[==========] Running 15 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260807 19:47:41.903542 6315 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.42.254:35443
I20260807 19:47:41.907565 6315 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:41.909727 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:47:41.944885 6326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:41.945254 6322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:41.946234 6315 server_base.cc:1061] running on GCE node
W20260807 19:47:41.945457 6323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:43.183091 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:47:43.183450 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:47:43.183660 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132063183633 us; error 0 us; skew 500 ppm
I20260807 19:47:43.185030 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:43.192479 6315 webserver.cc:533] Webserver started at http://127.6.42.254:34815/ using document root <none> and password file <none>
I20260807 19:47:43.194566 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:47:43.194859 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:43.195533 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:43.204635 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/master-0-root/instance:
uuid: "97201b96e55442659104305ba5f1eac0"
format_stamp: "Formatted at 2026-08-07 19:47:43 on dist-test-slave-lkx2"
I20260807 19:47:43.218973 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.013s user 0.011s sys 0.001s
I20260807 19:47:43.227756 6339 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:43.230482 6315 fs_manager.cc:730] Time spent opening block manager: real 0.007s user 0.005s sys 0.001s
I20260807 19:47:43.231122 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/master-0-root
uuid: "97201b96e55442659104305ba5f1eac0"
format_stamp: "Formatted at 2026-08-07 19:47:43 on dist-test-slave-lkx2"
I20260807 19:47:43.231575 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:43.291393 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:47:43.293525 6315 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:43.294207 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:43.389164 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.254:35443
I20260807 19:47:43.389302 6414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.254:35443 every 8 connection(s)
I20260807 19:47:43.404019 6415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:43.434041 6415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0: Bootstrap starting.
I20260807 19:47:43.443895 6415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:43.446983 6415 log.cc:826] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:43.455044 6415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0: No bootstrap required, opened a new log
I20260807 19:47:43.480278 6415 raft_consensus.cc:359] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97201b96e55442659104305ba5f1eac0" member_type: VOTER }
I20260807 19:47:43.481194 6415 raft_consensus.cc:385] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:43.481498 6415 raft_consensus.cc:740] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97201b96e55442659104305ba5f1eac0, State: Initialized, Role: FOLLOWER
I20260807 19:47:43.483265 6415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97201b96e55442659104305ba5f1eac0" member_type: VOTER }
I20260807 19:47:43.483913 6415 raft_consensus.cc:399] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:43.484247 6415 raft_consensus.cc:493] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:43.484671 6415 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:43.489334 6415 raft_consensus.cc:515] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97201b96e55442659104305ba5f1eac0" member_type: VOTER }
I20260807 19:47:43.490520 6415 leader_election.cc:304] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97201b96e55442659104305ba5f1eac0; no voters:
I20260807 19:47:43.492664 6415 leader_election.cc:290] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:43.493072 6420 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:43.495817 6420 raft_consensus.cc:697] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 1 LEADER]: Becoming Leader. State: Replica: 97201b96e55442659104305ba5f1eac0, State: Running, Role: LEADER
I20260807 19:47:43.497090 6420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97201b96e55442659104305ba5f1eac0" member_type: VOTER }
I20260807 19:47:43.498298 6415 sys_catalog.cc:565] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:43.512470 6421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97201b96e55442659104305ba5f1eac0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97201b96e55442659104305ba5f1eac0" member_type: VOTER } }
I20260807 19:47:43.512398 6423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97201b96e55442659104305ba5f1eac0. Latest consensus state: current_term: 1 leader_uuid: "97201b96e55442659104305ba5f1eac0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97201b96e55442659104305ba5f1eac0" member_type: VOTER } }
I20260807 19:47:43.513476 6421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:43.513688 6423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:43.517067 6435 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 19:47:43.531432 6435 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 19:47:43.534521 6315 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 19:47:43.554566 6435 catalog_manager.cc:1383] Generated new cluster ID: b19abe9ee7c64d1e813be6ed45dd5a39
I20260807 19:47:43.554941 6435 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 19:47:43.569515 6435 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 19:47:43.571676 6435 catalog_manager.cc:1540] Loading token signing keys...
I20260807 19:47:43.586817 6435 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0: Generated new TSK 0
I20260807 19:47:43.587996 6435 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 19:47:43.602031 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:47:43.612464 6446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:43.614086 6448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:43.618047 6452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:43.618623 6315 server_base.cc:1061] running on GCE node
I20260807 19:47:43.619415 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:47:43.619625 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:47:43.619788 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132063619770 us; error 0 us; skew 500 ppm
I20260807 19:47:43.620469 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:43.625048 6315 webserver.cc:533] Webserver started at http://127.6.42.193:41587/ using document root <none> and password file <none>
I20260807 19:47:43.625715 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:47:43.625945 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:43.626274 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:43.627609 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-0-root/instance:
uuid: "559e9c50afd348178dc1725870a64588"
format_stamp: "Formatted at 2026-08-07 19:47:43 on dist-test-slave-lkx2"
I20260807 19:47:43.634382 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 19:47:43.639124 6462 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:43.640143 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:47:43.640604 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-0-root
uuid: "559e9c50afd348178dc1725870a64588"
format_stamp: "Formatted at 2026-08-07 19:47:43 on dist-test-slave-lkx2"
I20260807 19:47:43.640929 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:43.685055 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:47:43.686592 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:43.689857 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:47:43.695228 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:43.695580 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:43.695859 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:47:43.696031 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:43.806339 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.193:34611
I20260807 19:47:43.806470 6550 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.193:34611 every 8 connection(s)
I20260807 19:47:43.814417 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:47:43.838367 6558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:43.850737 6559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:43.855618 6315 server_base.cc:1061] running on GCE node
W20260807 19:47:43.857352 6561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:43.858733 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:47:43.859021 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:47:43.859241 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132063859223 us; error 0 us; skew 500 ppm
I20260807 19:47:43.859845 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:43.863139 6315 webserver.cc:533] Webserver started at http://127.6.42.194:46833/ using document root <none> and password file <none>
I20260807 19:47:43.863698 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:47:43.863909 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:43.864189 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:43.865646 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-1-root/instance:
uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
format_stamp: "Formatted at 2026-08-07 19:47:43 on dist-test-slave-lkx2"
I20260807 19:47:43.870852 6551 heartbeater.cc:344] Connected to a master server at 127.6.42.254:35443
I20260807 19:47:43.871520 6551 heartbeater.cc:461] Registering TS with master...
I20260807 19:47:43.873131 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260807 19:47:43.873041 6551 heartbeater.cc:507] Master 127.6.42.254:35443 requested a full tablet report, sending...
I20260807 19:47:43.878341 6361 ts_manager.cc:194] Registered new tserver with Master: 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611)
I20260807 19:47:43.881517 6570 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:43.881984 6361 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59082
I20260807 19:47:43.882568 6315 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:47:43.882941 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-1-root
uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
format_stamp: "Formatted at 2026-08-07 19:47:43 on dist-test-slave-lkx2"
I20260807 19:47:43.883324 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:43.906978 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:47:43.908387 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:43.910960 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:47:43.915023 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:43.915243 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:43.915498 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:47:43.915683 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:44.021216 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.194:39551
I20260807 19:47:44.021335 6663 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.194:39551 every 8 connection(s)
I20260807 19:47:44.027979 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:47:44.036835 6669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:44.038285 6676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:44.046411 6665 heartbeater.cc:344] Connected to a master server at 127.6.42.254:35443
I20260807 19:47:44.046870 6665 heartbeater.cc:461] Registering TS with master...
I20260807 19:47:44.048027 6665 heartbeater.cc:507] Master 127.6.42.254:35443 requested a full tablet report, sending...
I20260807 19:47:44.050763 6361 ts_manager.cc:194] Registered new tserver with Master: f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
W20260807 19:47:44.052278 6678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:44.052700 6361 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59084
I20260807 19:47:44.053009 6315 server_base.cc:1061] running on GCE node
I20260807 19:47:44.054939 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:47:44.055207 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:47:44.055418 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132064055399 us; error 0 us; skew 500 ppm
I20260807 19:47:44.056157 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:44.059167 6315 webserver.cc:533] Webserver started at http://127.6.42.195:32943/ using document root <none> and password file <none>
I20260807 19:47:44.059819 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:47:44.060084 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:44.060436 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:44.061892 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-2-root/instance:
uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
format_stamp: "Formatted at 2026-08-07 19:47:44 on dist-test-slave-lkx2"
I20260807 19:47:44.068207 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:47:44.073184 6684 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:44.074039 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:44.074327 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-2-root
uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
format_stamp: "Formatted at 2026-08-07 19:47:44 on dist-test-slave-lkx2"
I20260807 19:47:44.074604 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786132061667097-6315-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:44.092753 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:47:44.094053 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:44.095757 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:47:44.098148 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:44.098361 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:44.098585 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:47:44.098752 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:44.198633 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.195:36983
I20260807 19:47:44.198734 6783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.195:36983 every 8 connection(s)
I20260807 19:47:44.221055 6784 heartbeater.cc:344] Connected to a master server at 127.6.42.254:35443
I20260807 19:47:44.221499 6784 heartbeater.cc:461] Registering TS with master...
I20260807 19:47:44.222393 6784 heartbeater.cc:507] Master 127.6.42.254:35443 requested a full tablet report, sending...
I20260807 19:47:44.224807 6361 ts_manager.cc:194] Registered new tserver with Master: b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:44.224962 6315 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013806185s
I20260807 19:47:44.226742 6361 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59096
I20260807 19:47:44.284638 6361 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59100:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:47:44.289980 6361 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:47:44.406736 6610 tablet_service.cc:1511] Processing CreateTablet for tablet 5916701f63fe4f61ade4abdd26f16786 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260807 19:47:44.408756 6610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5916701f63fe4f61ade4abdd26f16786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.416721 6719 tablet_service.cc:1511] Processing CreateTablet for tablet f68b6cd9b52544508dd3350010f8be02 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260807 19:47:44.418360 6719 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68b6cd9b52544508dd3350010f8be02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.418833 6720 tablet_service.cc:1511] Processing CreateTablet for tablet 5916701f63fe4f61ade4abdd26f16786 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260807 19:47:44.420259 6720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5916701f63fe4f61ade4abdd26f16786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.421726 6718 tablet_service.cc:1511] Processing CreateTablet for tablet df6aaeeb7b9b46abbfa682af10a1bf41 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260807 19:47:44.423234 6718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df6aaeeb7b9b46abbfa682af10a1bf41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.424510 6605 tablet_service.cc:1511] Processing CreateTablet for tablet 79c0e6d318974614931a9d66127a0fd1 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260807 19:47:44.425971 6605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c0e6d318974614931a9d66127a0fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.425742 6717 tablet_service.cc:1511] Processing CreateTablet for tablet f66e312e1c23422aa03e3d63485f351c (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260807 19:47:44.426177 6713 tablet_service.cc:1511] Processing CreateTablet for tablet cbe73bcb11b04d5d9dfea85d210f5516 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260807 19:47:44.427557 6717 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66e312e1c23422aa03e3d63485f351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.428383 6713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbe73bcb11b04d5d9dfea85d210f5516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.429836 6716 tablet_service.cc:1511] Processing CreateTablet for tablet 1685c500eccb455f8707900cf9763d5c (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260807 19:47:44.431330 6716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1685c500eccb455f8707900cf9763d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.433633 6712 tablet_service.cc:1511] Processing CreateTablet for tablet b8b716aaad814bc9ba10aeaed17309ad (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260807 19:47:44.435134 6712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b716aaad814bc9ba10aeaed17309ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.456969 6608 tablet_service.cc:1511] Processing CreateTablet for tablet df6aaeeb7b9b46abbfa682af10a1bf41 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260807 19:47:44.448886 6600 tablet_service.cc:1511] Processing CreateTablet for tablet b8b716aaad814bc9ba10aeaed17309ad (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260807 19:47:44.447947 6715 tablet_service.cc:1511] Processing CreateTablet for tablet 79c0e6d318974614931a9d66127a0fd1 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260807 19:47:44.461560 6608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df6aaeeb7b9b46abbfa682af10a1bf41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.461273 6607 tablet_service.cc:1511] Processing CreateTablet for tablet f66e312e1c23422aa03e3d63485f351c (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260807 19:47:44.458110 6609 tablet_service.cc:1511] Processing CreateTablet for tablet f68b6cd9b52544508dd3350010f8be02 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260807 19:47:44.461326 6602 tablet_service.cc:1511] Processing CreateTablet for tablet edc063e9b3c64636b307a0f06f78494e (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260807 19:47:44.465484 6601 tablet_service.cc:1511] Processing CreateTablet for tablet cbe73bcb11b04d5d9dfea85d210f5516 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260807 19:47:44.458206 6714 tablet_service.cc:1511] Processing CreateTablet for tablet edc063e9b3c64636b307a0f06f78494e (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260807 19:47:44.456861 6606 tablet_service.cc:1511] Processing CreateTablet for tablet 1685c500eccb455f8707900cf9763d5c (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260807 19:47:44.467024 6715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c0e6d318974614931a9d66127a0fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.470935 6600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b716aaad814bc9ba10aeaed17309ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.468528 6714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc063e9b3c64636b307a0f06f78494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.481945 6601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbe73bcb11b04d5d9dfea85d210f5516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.485100 6606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1685c500eccb455f8707900cf9763d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.495296 6602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc063e9b3c64636b307a0f06f78494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.497802 6607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66e312e1c23422aa03e3d63485f351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.500996 6609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68b6cd9b52544508dd3350010f8be02. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:47:44.534163 6434 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:47:44.569587 6500 tablet_service.cc:1511] Processing CreateTablet for tablet df6aaeeb7b9b46abbfa682af10a1bf41 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260807 19:47:44.567838 6807 tablet_bootstrap.cc:492] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:44.573181 6500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df6aaeeb7b9b46abbfa682af10a1bf41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.582358 6498 tablet_service.cc:1511] Processing CreateTablet for tablet f66e312e1c23422aa03e3d63485f351c (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260807 19:47:44.589489 6494 tablet_service.cc:1511] Processing CreateTablet for tablet 79c0e6d318974614931a9d66127a0fd1 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260807 19:47:44.591070 6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79c0e6d318974614931a9d66127a0fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.566434 6502 tablet_service.cc:1511] Processing CreateTablet for tablet f68b6cd9b52544508dd3350010f8be02 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260807 19:47:44.610745 6493 tablet_service.cc:1511] Processing CreateTablet for tablet edc063e9b3c64636b307a0f06f78494e (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260807 19:47:44.613574 6493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc063e9b3c64636b307a0f06f78494e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.619403 6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66e312e1c23422aa03e3d63485f351c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.619066 6504 tablet_service.cc:1511] Processing CreateTablet for tablet 5916701f63fe4f61ade4abdd26f16786 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260807 19:47:44.619052 6495 tablet_service.cc:1511] Processing CreateTablet for tablet 1685c500eccb455f8707900cf9763d5c (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260807 19:47:44.630148 6502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68b6cd9b52544508dd3350010f8be02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.627511 6490 tablet_service.cc:1511] Processing CreateTablet for tablet b8b716aaad814bc9ba10aeaed17309ad (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260807 19:47:44.649602 6492 tablet_service.cc:1511] Processing CreateTablet for tablet cbe73bcb11b04d5d9dfea85d210f5516 (DEFAULT_TABLE table=test-workload [id=66d2194246434425886373c6a00bb7cf]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260807 19:47:44.658360 6504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5916701f63fe4f61ade4abdd26f16786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.661633 6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1685c500eccb455f8707900cf9763d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.666208 6808 tablet_bootstrap.cc:492] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:44.672889 6490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b716aaad814bc9ba10aeaed17309ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.681546 6492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbe73bcb11b04d5d9dfea85d210f5516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:44.686446 6807 tablet_bootstrap.cc:654] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.732911 6808 tablet_bootstrap.cc:654] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.752729 6809 tablet_bootstrap.cc:492] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:44.755983 6808 tablet_bootstrap.cc:492] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:44.756546 6808 ts_tablet_manager.cc:1403] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.093s user 0.006s sys 0.012s
I20260807 19:47:44.757781 6807 tablet_bootstrap.cc:492] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:44.758355 6807 ts_tablet_manager.cc:1403] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.244s user 0.005s sys 0.024s
I20260807 19:47:44.759582 6808 raft_consensus.cc:359] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.760524 6808 raft_consensus.cc:385] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.760849 6808 raft_consensus.cc:740] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.761624 6808 consensus_queue.cc:260] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.762612 6809 tablet_bootstrap.cc:654] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.762521 6807 raft_consensus.cc:359] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:44.763434 6807 raft_consensus.cc:385] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.763787 6807 raft_consensus.cc:740] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.764652 6807 consensus_queue.cc:260] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:44.771165 6784 heartbeater.cc:499] Master 127.6.42.254:35443 was elected leader, sending a full tablet report...
I20260807 19:47:44.771440 6808 ts_tablet_manager.cc:1434] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260807 19:47:44.772733 6808 tablet_bootstrap.cc:492] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:44.773006 6809 tablet_bootstrap.cc:492] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:44.773464 6809 ts_tablet_manager.cc:1403] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.008s
I20260807 19:47:44.776661 6809 raft_consensus.cc:359] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.777447 6809 raft_consensus.cc:385] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.777757 6809 raft_consensus.cc:740] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.779062 6809 consensus_queue.cc:260] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.779814 6808 tablet_bootstrap.cc:654] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.780869 6807 ts_tablet_manager.cc:1434] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.022s user 0.006s sys 0.008s
I20260807 19:47:44.781641 6665 heartbeater.cc:499] Master 127.6.42.254:35443 was elected leader, sending a full tablet report...
I20260807 19:47:44.782608 6551 heartbeater.cc:499] Master 127.6.42.254:35443 was elected leader, sending a full tablet report...
I20260807 19:47:44.783092 6809 ts_tablet_manager.cc:1434] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.009s user 0.003s sys 0.007s
I20260807 19:47:44.784008 6809 tablet_bootstrap.cc:492] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:44.785686 6807 tablet_bootstrap.cc:492] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:44.797948 6809 tablet_bootstrap.cc:654] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.798302 6807 tablet_bootstrap.cc:654] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.802160 6808 tablet_bootstrap.cc:492] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:44.803335 6808 ts_tablet_manager.cc:1403] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.031s user 0.011s sys 0.004s
I20260807 19:47:44.809146 6809 tablet_bootstrap.cc:492] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:44.809679 6809 ts_tablet_manager.cc:1403] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.003s
I20260807 19:47:44.811064 6808 raft_consensus.cc:359] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:44.811919 6808 raft_consensus.cc:385] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.812273 6808 raft_consensus.cc:740] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.813059 6809 raft_consensus.cc:359] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.813915 6809 raft_consensus.cc:385] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.813102 6808 consensus_queue.cc:260] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:44.814216 6809 raft_consensus.cc:740] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.815104 6809 consensus_queue.cc:260] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
W20260807 19:47:44.816581 6552 tablet.cc:2406] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:44.817353 6809 ts_tablet_manager.cc:1434] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 19:47:44.818111 6808 ts_tablet_manager.cc:1434] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260807 19:47:44.819744 6808 tablet_bootstrap.cc:492] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:44.822561 6807 tablet_bootstrap.cc:492] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:44.823187 6807 ts_tablet_manager.cc:1403] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.038s user 0.015s sys 0.001s
I20260807 19:47:44.818269 6809 tablet_bootstrap.cc:492] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:44.826722 6807 raft_consensus.cc:359] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.827514 6808 tablet_bootstrap.cc:654] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.827574 6807 raft_consensus.cc:385] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.828111 6807 raft_consensus.cc:740] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.829033 6807 consensus_queue.cc:260] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.831629 6807 ts_tablet_manager.cc:1434] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:47:44.832644 6807 tablet_bootstrap.cc:492] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:44.834972 6809 tablet_bootstrap.cc:654] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.840124 6807 tablet_bootstrap.cc:654] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.842824 6808 tablet_bootstrap.cc:492] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:44.843376 6808 ts_tablet_manager.cc:1403] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.000s
I20260807 19:47:44.846897 6808 raft_consensus.cc:359] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.847767 6808 raft_consensus.cc:385] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.848110 6808 raft_consensus.cc:740] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.855036 6809 tablet_bootstrap.cc:492] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:44.855603 6809 ts_tablet_manager.cc:1403] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.004s
I20260807 19:47:44.858891 6809 raft_consensus.cc:359] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.859705 6809 raft_consensus.cc:385] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.860090 6809 raft_consensus.cc:740] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.860862 6809 consensus_queue.cc:260] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.861819 6808 consensus_queue.cc:260] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.866142 6809 ts_tablet_manager.cc:1434] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20260807 19:47:44.867249 6809 tablet_bootstrap.cc:492] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:44.874081 6809 tablet_bootstrap.cc:654] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.880172 6808 ts_tablet_manager.cc:1434] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20260807 19:47:44.889828 6808 tablet_bootstrap.cc:492] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:44.891758 6816 raft_consensus.cc:493] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:44.892306 6816 raft_consensus.cc:515] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.895117 6807 tablet_bootstrap.cc:492] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:44.895706 6807 ts_tablet_manager.cc:1403] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.063s user 0.018s sys 0.042s
I20260807 19:47:44.895788 6816 leader_election.cc:290] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:44.896437 6816 raft_consensus.cc:493] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:44.896914 6816 raft_consensus.cc:515] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.899029 6816 leader_election.cc:290] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:44.901060 6807 raft_consensus.cc:359] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.901854 6809 tablet_bootstrap.cc:492] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:44.901969 6807 raft_consensus.cc:385] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.902397 6809 ts_tablet_manager.cc:1403] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.035s user 0.015s sys 0.009s
I20260807 19:47:44.902493 6807 raft_consensus.cc:740] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.903982 6808 tablet_bootstrap.cc:654] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.903488 6807 consensus_queue.cc:260] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.910097 6807 ts_tablet_manager.cc:1434] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260807 19:47:44.911142 6807 tablet_bootstrap.cc:492] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:44.914131 6812 raft_consensus.cc:493] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:44.918434 6812 raft_consensus.cc:515] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.919617 6809 raft_consensus.cc:359] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.920442 6809 raft_consensus.cc:385] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:44.920734 6809 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:44.924952 6815 raft_consensus.cc:493] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:44.925532 6815 raft_consensus.cc:515] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.928438 6815 leader_election.cc:290] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:44.935827 6807 tablet_bootstrap.cc:654] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.921919 6812 leader_election.cc:290] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:44.936470 6809 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:44.946069 6809 ts_tablet_manager.cc:1434] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.043s user 0.006s sys 0.001s
I20260807 19:47:44.947245 6809 tablet_bootstrap.cc:492] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:44.977079 6809 tablet_bootstrap.cc:654] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:44.998111 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c0e6d318974614931a9d66127a0fd1" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
W20260807 19:47:45.001531 6573 leader_election.cc:343] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:44.994647 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
W20260807 19:47:45.004172 6688 leader_election.cc:343] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.008126 6628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.009569 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.009693 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
W20260807 19:47:45.010175 6465 leader_election.cc:343] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.010694 6742 raft_consensus.cc:2468] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 0.
W20260807 19:47:45.011456 6465 leader_election.cc:343] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.011808 6807 tablet_bootstrap.cc:492] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:45.012324 6807 ts_tablet_manager.cc:1403] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.101s user 0.023s sys 0.010s
I20260807 19:47:45.012212 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.013474 6743 raft_consensus.cc:2468] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 0.
I20260807 19:47:45.013566 6815 raft_consensus.cc:493] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.014341 6815 raft_consensus.cc:515] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.015622 6807 raft_consensus.cc:359] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.016604 6815 leader_election.cc:290] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.019999 6809 tablet_bootstrap.cc:492] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:45.017930 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.020748 6809 ts_tablet_manager.cc:1403] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.074s user 0.011s sys 0.004s
W20260807 19:47:45.021605 6573 leader_election.cc:343] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.023516 6464 leader_election.cc:304] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters: f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.023795 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.020030 6808 tablet_bootstrap.cc:492] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:45.016372 6807 raft_consensus.cc:385] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.024636 6832 raft_consensus.cc:2804] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.024814 6807 raft_consensus.cc:740] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.025091 6832 raft_consensus.cc:493] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.025525 6832 raft_consensus.cc:3060] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.023999 6809 raft_consensus.cc:359] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.026006 6809 raft_consensus.cc:385] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.025693 6807 consensus_queue.cc:260] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.026319 6809 raft_consensus.cc:740] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.027143 6809 consensus_queue.cc:260] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.024933 6808 ts_tablet_manager.cc:1403] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.135s user 0.020s sys 0.031s
I20260807 19:47:45.030481 6807 ts_tablet_manager.cc:1434] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260807 19:47:45.031596 6807 tablet_bootstrap.cc:492] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:45.025823 6464 leader_election.cc:304] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters: f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.035077 6816 raft_consensus.cc:2804] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.035337 6809 ts_tablet_manager.cc:1434] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260807 19:47:45.035499 6816 raft_consensus.cc:493] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.035869 6816 raft_consensus.cc:3060] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.036420 6809 tablet_bootstrap.cc:492] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:45.040398 6807 tablet_bootstrap.cc:654] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.040462 6808 raft_consensus.cc:359] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.041409 6808 raft_consensus.cc:385] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.041718 6808 raft_consensus.cc:740] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.042768 6808 consensus_queue.cc:260] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.044370 6816 raft_consensus.cc:515] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.045406 6808 ts_tablet_manager.cc:1434] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.016s user 0.005s sys 0.002s
I20260807 19:47:45.046485 6808 tablet_bootstrap.cc:492] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:45.048568 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:45.049419 6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
W20260807 19:47:45.050338 6465 leader_election.cc:343] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:47:45.052157 6577 leader_election.cc:343] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.052783 6577 leader_election.cc:304] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f4e49dcb89a24c59b04f2cbdc32abea2; no voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:47:45.054124 6808 tablet_bootstrap.cc:654] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.054360 6815 raft_consensus.cc:2749] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:47:45.048461 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c0e6d318974614931a9d66127a0fd1" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.055963 6515 raft_consensus.cc:2468] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 0.
I20260807 19:47:45.058822 6577 leader_election.cc:304] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters: b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:47:45.060588 6815 raft_consensus.cc:2804] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.060449 6832 raft_consensus.cc:515] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.061170 6815 raft_consensus.cc:493] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.061759 6815 raft_consensus.cc:3060] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.062116 6516 raft_consensus.cc:2393] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5a5eb44ea4c497fb18c88506aa9e758 in current term 1: Already voted for candidate 559e9c50afd348178dc1725870a64588 in this term.
I20260807 19:47:45.066566 6689 leader_election.cc:304] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758; no voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.067473 6812 raft_consensus.cc:3060] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.068406 6808 tablet_bootstrap.cc:492] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:45.068902 6808 ts_tablet_manager.cc:1403] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.008s
I20260807 19:47:45.072510 6808 raft_consensus.cc:359] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.073235 6808 raft_consensus.cc:385] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.073547 6809 tablet_bootstrap.cc:654] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.075420 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:45.076041 6743 raft_consensus.cc:3060] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.076390 6812 raft_consensus.cc:2749] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:47:45.073563 6808 raft_consensus.cc:740] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.079648 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:45.087873 6808 consensus_queue.cc:260] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
W20260807 19:47:45.089810 6465 leader_election.cc:343] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.090076 6816 leader_election.cc:290] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.080662 6832 leader_election.cc:290] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.081772 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:45.090624 6816 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.091202 6816 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.086637 6743 raft_consensus.cc:2468] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 1.
I20260807 19:47:45.095932 6464 leader_election.cc:304] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters: f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.096976 6832 raft_consensus.cc:2804] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.097718 6832 raft_consensus.cc:697] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:45.098515 6832 consensus_queue.cc:237] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.101691 6816 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.102874 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.103024 6742 raft_consensus.cc:2468] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 1.
I20260807 19:47:45.103071 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
W20260807 19:47:45.104620 6464 leader_election.cc:343] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:47:45.105096 6465 leader_election.cc:343] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.105360 6464 leader_election.cc:304] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters: f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.105561 6465 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 559e9c50afd348178dc1725870a64588; no voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.104485 6815 raft_consensus.cc:515] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.106719 6816 raft_consensus.cc:2804] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.107367 6809 tablet_bootstrap.cc:492] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:45.107836 6808 ts_tablet_manager.cc:1434] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.039s user 0.010s sys 0.000s
I20260807 19:47:45.109086 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c0e6d318974614931a9d66127a0fd1" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:45.109721 6516 raft_consensus.cc:3060] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.110687 6808 tablet_bootstrap.cc:492] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:45.115797 6815 leader_election.cc:290] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.116950 6807 tablet_bootstrap.cc:492] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:45.117003 6516 raft_consensus.cc:2468] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 1.
I20260807 19:47:45.117172 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79c0e6d318974614931a9d66127a0fd1" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:45.117519 6807 ts_tablet_manager.cc:1403] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.086s user 0.031s sys 0.028s
I20260807 19:47:45.117833 6743 raft_consensus.cc:3060] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.118526 6577 leader_election.cc:304] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.119402 6815 raft_consensus.cc:2804] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.120056 6809 ts_tablet_manager.cc:1403] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.084s user 0.009s sys 0.005s
I20260807 19:47:45.120755 6807 raft_consensus.cc:359] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.121369 6807 raft_consensus.cc:385] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.121629 6807 raft_consensus.cc:740] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.122573 6807 consensus_queue.cc:260] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.123167 6809 raft_consensus.cc:359] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:45.123941 6809 raft_consensus.cc:385] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.124236 6809 raft_consensus.cc:740] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.127215 6816 raft_consensus.cc:697] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:45.133211 6743 raft_consensus.cc:2468] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 1.
I20260807 19:47:45.131433 6809 consensus_queue.cc:260] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:45.136063 6809 ts_tablet_manager.cc:1434] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260807 19:47:45.138895 6807 ts_tablet_manager.cc:1434] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.021s user 0.003s sys 0.005s
I20260807 19:47:45.140225 6809 tablet_bootstrap.cc:492] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:45.139621 6816 consensus_queue.cc:237] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.129611 6832 raft_consensus.cc:2749] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:47:45.141546 6355 catalog_manager.cc:5697] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:45.146672 6808 tablet_bootstrap.cc:654] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.145980 6815 raft_consensus.cc:697] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:45.145605 6807 tablet_bootstrap.cc:492] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:45.150385 6815 consensus_queue.cc:237] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.155290 6871 raft_consensus.cc:493] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.156284 6871 raft_consensus.cc:515] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:45.163786 6871 leader_election.cc:290] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983), 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611)
I20260807 19:47:45.165136 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.165616 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.166203 6743 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 0.
I20260807 19:47:45.166626 6516 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 0.
I20260807 19:47:45.168066 6573 leader_election.cc:304] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.174443 6807 tablet_bootstrap.cc:654] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.181030 6815 raft_consensus.cc:2804] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.181505 6815 raft_consensus.cc:493] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.181856 6815 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.185176 6809 tablet_bootstrap.cc:654] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.179600 6361 catalog_manager.cc:5697] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:45.193064 6864 raft_consensus.cc:493] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.193715 6864 raft_consensus.cc:515] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.198480 6815 raft_consensus.cc:515] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:45.199620 6864 leader_election.cc:290] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:45.203549 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:45.203920 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:45.204241 6743 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.204502 6515 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.201268 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b716aaad814bc9ba10aeaed17309ad" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.209100 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b716aaad814bc9ba10aeaed17309ad" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.202729 6815 leader_election.cc:290] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 election: Requested vote from peers b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983), 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611)
I20260807 19:47:45.208021 6629 raft_consensus.cc:2468] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 0.
W20260807 19:47:45.213910 6689 leader_election.cc:343] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.217583 6808 tablet_bootstrap.cc:492] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:45.218225 6808 ts_tablet_manager.cc:1403] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.108s user 0.029s sys 0.009s
I20260807 19:47:45.218031 6688 leader_election.cc:304] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters: 559e9c50afd348178dc1725870a64588
I20260807 19:47:45.219899 6515 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 1.
I20260807 19:47:45.223264 6864 raft_consensus.cc:2804] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.223680 6864 raft_consensus.cc:493] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.224017 6864 raft_consensus.cc:3060] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.225004 6577 leader_election.cc:304] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.225968 6815 raft_consensus.cc:2804] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.226598 6815 raft_consensus.cc:697] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:45.227627 6815 consensus_queue.cc:237] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:45.232683 6809 tablet_bootstrap.cc:492] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:45.233201 6809 ts_tablet_manager.cc:1403] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.093s user 0.013s sys 0.005s
I20260807 19:47:45.230406 6808 raft_consensus.cc:359] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.234457 6808 raft_consensus.cc:385] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.235566 6864 raft_consensus.cc:515] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.236116 6743 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 1.
I20260807 19:47:45.237078 6807 tablet_bootstrap.cc:492] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:45.237699 6807 ts_tablet_manager.cc:1403] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.095s user 0.016s sys 0.007s
I20260807 19:47:45.237756 6864 leader_election.cc:290] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:45.239665 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b716aaad814bc9ba10aeaed17309ad" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:45.234823 6808 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.240450 6629 raft_consensus.cc:3060] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.246938 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b716aaad814bc9ba10aeaed17309ad" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:45.245867 6808 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.248381 6807 raft_consensus.cc:359] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
W20260807 19:47:45.257939 6689 leader_election.cc:343] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.252188 6809 raft_consensus.cc:359] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.251084 6629 raft_consensus.cc:2468] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 1.
I20260807 19:47:45.258764 6809 raft_consensus.cc:385] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.259121 6809 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.259737 6688 leader_election.cc:304] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters: 559e9c50afd348178dc1725870a64588
I20260807 19:47:45.260830 6864 raft_consensus.cc:2804] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.261363 6808 ts_tablet_manager.cc:1434] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.043s user 0.009s sys 0.000s
I20260807 19:47:45.261621 6864 raft_consensus.cc:697] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:45.262487 6808 tablet_bootstrap.cc:492] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:45.262485 6864 consensus_queue.cc:237] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.257952 6807 raft_consensus.cc:385] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.267033 6807 raft_consensus.cc:740] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.267874 6807 consensus_queue.cc:260] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.275982 6809 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.278348 6808 tablet_bootstrap.cc:654] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.285841 6809 ts_tablet_manager.cc:1434] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.052s user 0.007s sys 0.003s
I20260807 19:47:45.287039 6809 tablet_bootstrap.cc:492] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:45.288465 6355 catalog_manager.cc:5697] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:45.290632 6807 ts_tablet_manager.cc:1434] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.052s user 0.008s sys 0.002s
I20260807 19:47:45.291657 6807 tablet_bootstrap.cc:492] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:45.298023 6358 catalog_manager.cc:5697] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:47:45.302819 6361 catalog_manager.cc:5697] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:45.298025 6807 tablet_bootstrap.cc:654] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.316998 6809 tablet_bootstrap.cc:654] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.317168 6808 tablet_bootstrap.cc:492] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:45.318017 6808 ts_tablet_manager.cc:1403] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.056s user 0.018s sys 0.010s
I20260807 19:47:45.323750 6808 raft_consensus.cc:359] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.326040 6807 tablet_bootstrap.cc:492] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:45.326619 6807 ts_tablet_manager.cc:1403] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.035s user 0.015s sys 0.004s
I20260807 19:47:45.329742 6807 raft_consensus.cc:359] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.330498 6807 raft_consensus.cc:385] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.330813 6807 raft_consensus.cc:740] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.331812 6807 consensus_queue.cc:260] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.334694 6807 ts_tablet_manager.cc:1434] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260807 19:47:45.335804 6807 tablet_bootstrap.cc:492] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:45.324754 6808 raft_consensus.cc:385] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.337276 6808 raft_consensus.cc:740] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.338099 6808 consensus_queue.cc:260] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.340549 6808 ts_tablet_manager.cc:1434] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.022s user 0.006s sys 0.004s
I20260807 19:47:45.341598 6808 tablet_bootstrap.cc:492] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:45.343494 6807 tablet_bootstrap.cc:654] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.345183 6815 raft_consensus.cc:493] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.345824 6815 raft_consensus.cc:515] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.348305 6815 leader_election.cc:290] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.348811 6808 tablet_bootstrap.cc:654] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.350636 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.351577 6743 raft_consensus.cc:2393] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f4e49dcb89a24c59b04f2cbdc32abea2 in current term 1: Already voted for candidate 559e9c50afd348178dc1725870a64588 in this term.
I20260807 19:47:45.351575 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.355597 6573 leader_election.cc:304] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f4e49dcb89a24c59b04f2cbdc32abea2; no voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:47:45.357246 6815 raft_consensus.cc:3060] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.362797 6809 tablet_bootstrap.cc:492] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:45.363435 6809 ts_tablet_manager.cc:1403] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.011s
I20260807 19:47:45.366415 6809 raft_consensus.cc:359] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.367256 6809 raft_consensus.cc:385] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.367600 6809 raft_consensus.cc:740] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.368256 6815 raft_consensus.cc:2749] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:47:45.368476 6809 consensus_queue.cc:260] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.371727 6809 ts_tablet_manager.cc:1434] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:47:45.371815 6807 tablet_bootstrap.cc:492] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:45.372558 6807 ts_tablet_manager.cc:1403] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.001s
I20260807 19:47:45.372957 6809 tablet_bootstrap.cc:492] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588: Bootstrap starting.
I20260807 19:47:45.375587 6816 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.376137 6816 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.378400 6816 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.379269 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.380229 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
W20260807 19:47:45.380666 6465 leader_election.cc:343] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:47:45.381789 6464 leader_election.cc:343] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:47:45.382246 6464 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 559e9c50afd348178dc1725870a64588; no voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.383327 6816 raft_consensus.cc:2749] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:47:45.383580 6832 raft_consensus.cc:493] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.384370 6832 raft_consensus.cc:515] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.375599 6807 raft_consensus.cc:359] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.385955 6807 raft_consensus.cc:385] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.386222 6807 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.386576 6832 leader_election.cc:290] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.387125 6808 tablet_bootstrap.cc:492] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:45.386907 6807 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.387709 6808 ts_tablet_manager.cc:1403] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.046s user 0.014s sys 0.017s
I20260807 19:47:45.389263 6807 ts_tablet_manager.cc:1434] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.016s user 0.004s sys 0.001s
I20260807 19:47:45.390168 6807 tablet_bootstrap.cc:492] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: Bootstrap starting.
I20260807 19:47:45.390694 6808 raft_consensus.cc:359] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.391414 6808 raft_consensus.cc:385] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.391746 6808 raft_consensus.cc:740] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.393252 6808 consensus_queue.cc:260] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.395800 6808 ts_tablet_manager.cc:1434] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.008s user 0.005s sys 0.001s
I20260807 19:47:45.396868 6808 tablet_bootstrap.cc:492] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: Bootstrap starting.
I20260807 19:47:45.398239 6807 tablet_bootstrap.cc:654] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.401773 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.402515 6743 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 0.
I20260807 19:47:45.404048 6808 tablet_bootstrap.cc:654] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.390308 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.405282 6464 leader_election.cc:304] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:47:45.405613 6629 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 0.
I20260807 19:47:45.406401 6832 raft_consensus.cc:2804] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.406802 6832 raft_consensus.cc:493] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.407152 6832 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.414424 6832 raft_consensus.cc:515] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.417481 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:45.418097 6629 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.418931 6832 leader_election.cc:290] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.420127 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:45.420739 6743 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.422091 6808 tablet_bootstrap.cc:492] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: No bootstrap required, opened a new log
I20260807 19:47:45.423869 6808 ts_tablet_manager.cc:1403] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.008s
I20260807 19:47:45.426679 6629 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 1.
I20260807 19:47:45.427350 6743 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 1.
I20260807 19:47:45.426878 6808 raft_consensus.cc:359] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.428009 6808 raft_consensus.cc:385] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.428157 6465 leader_election.cc:304] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.428382 6808 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.383145 6809 tablet_bootstrap.cc:654] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:45.429348 6808 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.432284 6808 ts_tablet_manager.cc:1434] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260807 19:47:45.435396 6832 raft_consensus.cc:2804] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.435986 6832 raft_consensus.cc:697] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:45.436873 6832 consensus_queue.cc:237] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.441614 6807 tablet_bootstrap.cc:492] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: No bootstrap required, opened a new log
I20260807 19:47:45.442245 6807 ts_tablet_manager.cc:1403] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent bootstrapping tablet: real 0.052s user 0.008s sys 0.015s
I20260807 19:47:45.446269 6807 raft_consensus.cc:359] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.447108 6807 raft_consensus.cc:385] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.447404 6807 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.447970 6807 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.448575 6809 tablet_bootstrap.cc:492] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588: No bootstrap required, opened a new log
I20260807 19:47:45.449589 6807 ts_tablet_manager.cc:1434] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:47:45.451154 6809 ts_tablet_manager.cc:1403] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588: Time spent bootstrapping tablet: real 0.078s user 0.014s sys 0.000s
I20260807 19:47:45.454021 6809 raft_consensus.cc:359] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.454704 6809 raft_consensus.cc:385] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:45.454991 6809 raft_consensus.cc:740] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Initialized, Role: FOLLOWER
I20260807 19:47:45.455826 6809 consensus_queue.cc:260] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.459997 6809 ts_tablet_manager.cc:1434] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:47:45.461536 6361 catalog_manager.cc:5697] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 0 to 1, leader changed from <none> to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
W20260807 19:47:45.540174 6434 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:47:45.577534 6864 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.578038 6864 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.579790 6864 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:45.580766 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.581173 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.581488 6515 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 0.
I20260807 19:47:45.581830 6629 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 0.
I20260807 19:47:45.582791 6689 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:47:45.583484 6864 raft_consensus.cc:2804] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.583833 6864 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.584153 6864 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.588836 6864 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.590545 6864 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:45.591531 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:45.591691 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:45.592213 6629 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.592170 6515 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.596923 6629 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 1.
I20260807 19:47:45.597998 6688 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.598686 6864 raft_consensus.cc:2804] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.598984 6515 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 1.
I20260807 19:47:45.599097 6864 raft_consensus.cc:697] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:45.600505 6864 consensus_queue.cc:237] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.608646 6361 catalog_manager.cc:5697] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:47:45.614519 6832 consensus_queue.cc:1048] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.625015 6629 raft_consensus.cc:3060] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.637223 6864 raft_consensus.cc:493] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.637799 6864 raft_consensus.cc:515] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.640093 6864 leader_election.cc:290] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:45.640556 6868 consensus_queue.cc:1048] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:47:45.643664 6832 raft_consensus.cc:493] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.644784 6832 raft_consensus.cc:515] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.645995 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.648695 6816 consensus_queue.cc:1048] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.649174 6629 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 0.
I20260807 19:47:45.650767 6832 leader_election.cc:290] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.648077 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.652858 6515 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 0.
I20260807 19:47:45.653501 6628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b716aaad814bc9ba10aeaed17309ad" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" is_pre_election: true
I20260807 19:47:45.650686 6688 leader_election.cc:304] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.654284 6628 raft_consensus.cc:2393] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 559e9c50afd348178dc1725870a64588 in current term 1: Already voted for candidate b5a5eb44ea4c497fb18c88506aa9e758 in this term.
I20260807 19:47:45.655355 6864 raft_consensus.cc:2804] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.655781 6864 raft_consensus.cc:493] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.656145 6864 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.660737 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b716aaad814bc9ba10aeaed17309ad" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.662741 6464 leader_election.cc:304] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 559e9c50afd348178dc1725870a64588; no voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:45.668828 6864 raft_consensus.cc:515] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.672071 6832 raft_consensus.cc:3060] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.672685 6628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:45.673350 6628 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.672681 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:45.680639 6515 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.681264 6832 raft_consensus.cc:2749] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:47:45.676681 6864 leader_election.cc:290] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:45.686492 6628 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 1.
I20260807 19:47:45.688674 6688 leader_election.cc:304] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.691412 6864 raft_consensus.cc:2804] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.691948 6864 raft_consensus.cc:697] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:45.692870 6864 consensus_queue.cc:237] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.698145 6832 consensus_queue.cc:1048] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.708242 6515 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 1.
I20260807 19:47:45.709988 6361 catalog_manager.cc:5697] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:47:45.754509 6864 consensus_queue.cc:1048] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.763764 6815 consensus_queue.cc:1048] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.770403 6864 consensus_queue.cc:1048] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.780681 6815 consensus_queue.cc:1048] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.802094 6928 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:45.804150 6928 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.823729 6815 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.828373 6932 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:47:45.841987 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588" is_pre_election: true
I20260807 19:47:45.842723 6516 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 0.
I20260807 19:47:45.844043 6577 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.845125 6935 raft_consensus.cc:2804] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:45.845512 6935 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:45.845829 6935 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.843883 6928 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.852265 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" is_pre_election: true
I20260807 19:47:45.853947 6742 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 0.
I20260807 19:47:45.863991 6935 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.866987 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:45.867638 6516 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.872735 6935 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:45.873104 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:45.873807 6743 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:45.876914 6516 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 1.
I20260807 19:47:45.878362 6577 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:45.879393 6935 raft_consensus.cc:2804] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:45.879966 6935 raft_consensus.cc:697] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:45.881175 6743 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 1.
I20260807 19:47:45.880928 6935 consensus_queue.cc:237] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:45.890600 6356 catalog_manager.cc:5697] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:45.905297 6832 consensus_queue.cc:1048] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:45.909745 6907 consensus_queue.cc:1048] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:47:46.018141 6812 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:46.030405 6812 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:46.123335 6812 consensus_queue.cc:1048] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:46.136315 6812 consensus_queue.cc:1048] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:47:46.241283 6629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:36094
I20260807 19:47:46.242022 6629 raft_consensus.cc:606] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Received request to transfer leadership to TS 559e9c50afd348178dc1725870a64588
I20260807 19:47:46.252156 6629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:36094
I20260807 19:47:46.252903 6629 raft_consensus.cc:606] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Received request to transfer leadership to TS 559e9c50afd348178dc1725870a64588
I20260807 19:47:46.271521 6516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c"
dest_uuid: "559e9c50afd348178dc1725870a64588"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:33182
I20260807 19:47:46.272073 6516 raft_consensus.cc:606] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:46.292526 6743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:34378
I20260807 19:47:46.293077 6743 raft_consensus.cc:606] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:46.299227 6743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:34378
I20260807 19:47:46.299924 6743 raft_consensus.cc:606] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:46.310135 6932 raft_consensus.cc:993] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:47:46.310664 6935 raft_consensus.cc:1081] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Signalling peer 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:47:46.312687 6516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786"
dest_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:39056
I20260807 19:47:46.313230 6516 raft_consensus.cc:493] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:46.313541 6516 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.318686 6516 raft_consensus.cc:515] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:46.320793 6516 leader_election.cc:290] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 2 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:46.321803 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:46.322165 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:46.322554 6629 raft_consensus.cc:3055] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:46.322870 6743 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.322945 6629 raft_consensus.cc:740] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:46.324265 6629 consensus_queue.cc:260] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:46.326073 6629 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.330725 6743 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 2.
I20260807 19:47:46.330811 6935 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:47:46.332576 6464 leader_election.cc:304] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:47:46.333452 6907 raft_consensus.cc:2804] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:46.334003 6907 raft_consensus.cc:697] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:46.334928 6907 consensus_queue.cc:237] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:47:46.339986 6629 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 2.
I20260807 19:47:46.345661 6932 raft_consensus.cc:993] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:47:46.346182 6928 raft_consensus.cc:1081] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Signalling peer 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:47:46.346465 6358 catalog_manager.cc:5697] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 1 to 2, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } }
I20260807 19:47:46.353957 6516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:39056
I20260807 19:47:46.354746 6516 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:46.355284 6516 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.358142 6925 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:46.365442 6516 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.370266 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:46.371006 6629 raft_consensus.cc:3055] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:46.371367 6629 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:46.371794 6516 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 2 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:46.372267 6629 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.376021 6629 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.377048 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:46.383404 6629 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 2.
I20260807 19:47:46.384850 6465 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters: b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:47:46.386154 6907 raft_consensus.cc:2804] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:46.386722 6907 raft_consensus.cc:697] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:46.387619 6907 consensus_queue.cc:237] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.396826 6358 catalog_manager.cc:5697] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 1 to 2, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:46.410374 6868 raft_consensus.cc:993] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:46.410959 6907 raft_consensus.cc:1081] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:46.412575 6629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59796
I20260807 19:47:46.413074 6629 raft_consensus.cc:493] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:46.413362 6629 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.419857 6629 raft_consensus.cc:515] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.421864 6629 leader_election.cc:290] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:46.423527 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:46.423527 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:46.424206 6742 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.424543 6516 raft_consensus.cc:3055] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:46.424870 6516 raft_consensus.cc:740] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:46.425841 6516 consensus_queue.cc:260] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.427143 6516 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.431682 6742 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:46.433092 6573 leader_election.cc:304] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:46.434022 6925 raft_consensus.cc:2804] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:46.434556 6925 raft_consensus.cc:697] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:46.435411 6925 consensus_queue.cc:237] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.436208 6516 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:46.445338 6864 raft_consensus.cc:993] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:46.445794 6920 raft_consensus.cc:1081] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:46.446427 6358 catalog_manager.cc:5697] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 1 to 2, leader changed from 559e9c50afd348178dc1725870a64588 (127.6.42.193) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:46.447729 6629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59800
I20260807 19:47:46.448401 6629 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:46.448792 6629 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.456117 6629 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.458482 6629 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:46.459470 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:46.460094 6516 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.459933 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:46.460683 6742 raft_consensus.cc:3055] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:46.461031 6742 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:46.461876 6742 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.463171 6742 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.465533 6516 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:46.466739 6577 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:46.467484 6925 raft_consensus.cc:2804] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:46.467949 6925 raft_consensus.cc:697] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:46.468727 6925 consensus_queue.cc:237] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.470408 6742 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:46.476253 6358 catalog_manager.cc:5697] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 1 to 2, leader changed from b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:46.540619 6864 raft_consensus.cc:993] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:46.541152 6920 raft_consensus.cc:1081] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:46.543149 6629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59800
I20260807 19:47:46.543763 6629 raft_consensus.cc:493] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:46.544159 6629 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.551290 6629 raft_consensus.cc:515] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.553628 6629 leader_election.cc:290] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:46.555162 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:46.555835 6742 raft_consensus.cc:3055] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:46.556051 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:46.556216 6742 raft_consensus.cc:740] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:46.556771 6516 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.557286 6742 consensus_queue.cc:260] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.558600 6742 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.562724 6516 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:46.563849 6577 leader_election.cc:304] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:46.563999 6742 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:46.564626 6925 raft_consensus.cc:2804] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:46.565152 6925 raft_consensus.cc:697] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:46.565901 6925 consensus_queue.cc:237] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:46.572376 6358 catalog_manager.cc:5697] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 1 to 2, leader changed from b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:46.719940 6742 raft_consensus.cc:1275] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.721443 6907 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.728839 6629 raft_consensus.cc:1275] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.731271 6832 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.827402 6742 raft_consensus.cc:1275] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.828694 6925 consensus_queue.cc:1048] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.836576 6516 raft_consensus.cc:1275] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.838441 6932 consensus_queue.cc:1048] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.854537 6516 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.855929 6932 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.864521 6742 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.865964 6928 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.882393 6629 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.883596 6868 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.890743 6742 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:46.897094 6742 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.898308 6868 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.984974 6516 raft_consensus.cc:1275] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.986440 6928 consensus_queue.cc:1048] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:46.993875 6742 raft_consensus.cc:1275] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:47:46.995373 6925 consensus_queue.cc:1048] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:47:49.338335 6315 auto_leader_rebalancer-test.cc:402] The leader distribution is
I20260807 19:47:49.339408 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:49.361622 6629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:36108
I20260807 19:47:49.362144 6629 raft_consensus.cc:606] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Received request to transfer leadership to TS b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:47:49.374094 6516 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "559e9c50afd348178dc1725870a64588"
mode: GRACEFUL
new_leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:33192
I20260807 19:47:49.374737 6516 raft_consensus.cc:606] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Received request to transfer leadership to TS b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:47:49.376044 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 19:47:49.376116 6964 raft_consensus.cc:993] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:47:49.376722 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:49.376722 6932 raft_consensus.cc:1081] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Signalling peer b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:47:49.378435 6743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:53338
I20260807 19:47:49.379083 6743 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:49.379487 6743 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:49.384145 6743 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:49.386420 6743 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 3 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:49.386860 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:49.387039 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:49.387477 6516 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:49.387617 6629 raft_consensus.cc:3055] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:47:49.387986 6629 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:49.388659 6629 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:49.389572 6629 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:49.392546 6516 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 3.
I20260807 19:47:49.393760 6629 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 3.
I20260807 19:47:49.393761 6689 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:47:49.394726 6920 raft_consensus.cc:2804] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:47:49.395903 6920 raft_consensus.cc:697] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:49.396736 6920 consensus_queue.cc:237] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:49.406181 6356 catalog_manager.cc:5697] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 2 to 3, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:47:49.628625 6868 raft_consensus.cc:993] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: : Instructing follower b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:47:49.629053 6982 raft_consensus.cc:1081] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Signalling peer b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:47:49.630647 6743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:53336
I20260807 19:47:49.631132 6743 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:49.631443 6743 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:49.636524 6743 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:49.638154 6743 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 3 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:47:49.638903 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:49.639185 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:47:49.639536 6516 raft_consensus.cc:3055] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:47:49.639686 6629 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:49.639856 6516 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:49.640631 6516 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:49.641530 6516 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:47:49.644003 6629 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 3.
I20260807 19:47:49.644973 6688 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:49.645671 6920 raft_consensus.cc:2804] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:47:49.646183 6920 raft_consensus.cc:697] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:49.646894 6516 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 3.
I20260807 19:47:49.647013 6920 consensus_queue.cc:237] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:49.653848 6356 catalog_manager.cc:5697] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 2 to 3, leader changed from 559e9c50afd348178dc1725870a64588 (127.6.42.193) to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:47:49.882187 6629 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:47:49.883469 6920 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:47:49.895750 6516 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:47:49.897207 6920 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:47:50.071734 6516 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:47:50.073246 6920 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:47:50.082159 6629 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:47:50.084111 6965 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:47:50.378108 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:50.383880 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:50.384387 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:51.385269 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:51.391597 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:51.392057 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:52.393393 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:52.399991 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:52.400619 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:53.401919 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:53.406813 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:53.407326 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:54.408563 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:54.414011 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:54.414456 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:55.415555 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:55.420445 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:55.420862 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:56.421846 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:56.427043 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:56.427557 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:57.428517 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:57.434037 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:57.434465 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:58.435427 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:47:58.440412 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:47:58.440902 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:47:59.471803 6315 auto_leader_rebalancer-test.cc:421] The leader distribution is
I20260807 19:47:59.477075 6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786"
dest_uuid: "559e9c50afd348178dc1725870a64588"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:33182
I20260807 19:47:59.477808 6515 raft_consensus.cc:606] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:59.483461 6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02"
dest_uuid: "559e9c50afd348178dc1725870a64588"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:33182
I20260807 19:47:59.484036 6515 raft_consensus.cc:606] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:59.489437 6743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:34378
I20260807 19:47:59.489951 6743 raft_consensus.cc:606] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:59.495507 6515 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c"
dest_uuid: "559e9c50afd348178dc1725870a64588"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:33182
I20260807 19:47:59.496057 6515 raft_consensus.cc:606] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:59.510967 6743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
mode: GRACEFUL
new_leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:34378
I20260807 19:47:59.511456 6743 raft_consensus.cc:606] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Received request to transfer leadership to TS f4e49dcb89a24c59b04f2cbdc32abea2
I20260807 19:47:59.568140 6982 raft_consensus.cc:993] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.568847 7008 raft_consensus.cc:1081] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.570638 6629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59796
I20260807 19:47:59.571246 6629 raft_consensus.cc:493] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:59.571734 6629 raft_consensus.cc:3060] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:59.572961 6920 raft_consensus.cc:993] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.573881 7009 raft_consensus.cc:1081] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.576241 6627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59800
I20260807 19:47:59.576982 6627 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:59.577241 6627 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:47:59.578976 6629 raft_consensus.cc:515] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.580904 6629 leader_election.cc:290] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:59.582307 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:59.582275 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68b6cd9b52544508dd3350010f8be02" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:59.583185 6743 raft_consensus.cc:3060] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:59.583303 6515 raft_consensus.cc:3055] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:59.583653 6515 raft_consensus.cc:740] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:59.583973 6627 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.584427 6515 consensus_queue.cc:260] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.585819 6515 raft_consensus.cc:3060] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:59.586274 6627 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 4 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:59.587846 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:59.588555 6742 raft_consensus.cc:3055] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:47:59.588907 6742 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:59.589666 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:59.590241 6516 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:47:59.590343 6743 raft_consensus.cc:2468] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:59.589855 6742 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.591552 6742 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:47:59.593457 6515 raft_consensus.cc:2468] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:59.594749 6573 leader_election.cc:304] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:59.595552 6516 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 4.
I20260807 19:47:59.596246 7000 raft_consensus.cc:2804] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:59.596915 6577 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:59.597543 7000 raft_consensus.cc:697] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:59.598876 7010 raft_consensus.cc:2804] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:47:59.598781 7000 consensus_queue.cc:237] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.598807 6742 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 4.
I20260807 19:47:59.599711 7010 raft_consensus.cc:697] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:59.600898 7010 consensus_queue.cc:237] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.613629 6358 catalog_manager.cc:5697] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 1 to 2, leader changed from 559e9c50afd348178dc1725870a64588 (127.6.42.193) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:59.631541 6358 catalog_manager.cc:5697] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 3 to 4, leader changed from b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 4 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:59.763942 6920 raft_consensus.cc:993] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.764389 7009 raft_consensus.cc:1081] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.765807 6627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59800
I20260807 19:47:59.766288 6627 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:59.766561 6627 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:47:59.771246 6627 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.772964 6627 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 4 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:59.773875 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:59.774024 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:59.774478 6516 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:47:59.774567 6742 raft_consensus.cc:3055] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:47:59.774895 6742 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:47:59.775570 6742 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.776676 6742 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:47:59.778946 6516 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 4.
I20260807 19:47:59.780097 6577 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:59.780984 7000 raft_consensus.cc:2804] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:47:59.781554 7000 raft_consensus.cc:697] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:59.782068 6742 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 4.
I20260807 19:47:59.782598 7000 consensus_queue.cc:237] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.791812 6358 catalog_manager.cc:5697] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 3 to 4, leader changed from b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 4 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:47:59.844496 6982 raft_consensus.cc:993] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.844971 7008 raft_consensus.cc:1081] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:47:59.846549 6627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59796
I20260807 19:47:59.847064 6627 raft_consensus.cc:493] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:59.847362 6627 raft_consensus.cc:3060] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:59.852197 6627 raft_consensus.cc:515] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.854251 6627 leader_election.cc:290] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:47:59.855181 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:47:59.855418 6742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f66e312e1c23422aa03e3d63485f351c" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:47:59.855834 6516 raft_consensus.cc:3055] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:47:59.856117 6742 raft_consensus.cc:3060] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:59.856242 6516 raft_consensus.cc:740] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:47:59.857015 6516 consensus_queue.cc:260] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.857884 6516 raft_consensus.cc:3060] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:47:59.861282 6742 raft_consensus.cc:2468] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:59.862239 6516 raft_consensus.cc:2468] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 2.
I20260807 19:47:59.862468 6573 leader_election.cc:304] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:47:59.863334 7000 raft_consensus.cc:2804] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:47:59.863811 7000 raft_consensus.cc:697] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:47:59.864650 7000 consensus_queue.cc:237] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:47:59.872172 6358 catalog_manager.cc:5697] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 1 to 2, leader changed from 559e9c50afd348178dc1725870a64588 (127.6.42.193) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:00.028896 6516 raft_consensus.cc:1275] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:00.030249 6982 raft_consensus.cc:993] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: : Instructing follower f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:48:00.030136 7000 consensus_queue.cc:1048] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:48:00.030828 7008 raft_consensus.cc:1081] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Signalling peer f4e49dcb89a24c59b04f2cbdc32abea2 to start an election
I20260807 19:48:00.032295 6627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
from {username='slave'} at 127.0.0.1:59796
I20260807 19:48:00.032788 6627 raft_consensus.cc:493] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:00.033145 6627 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:00.040299 6627 raft_consensus.cc:515] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:48:00.043833 6742 raft_consensus.cc:1275] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:00.044973 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:48:00.045562 6743 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:00.046339 6627 leader_election.cc:290] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 3 election: Requested vote from peers b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983), 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611)
I20260807 19:48:00.046947 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:48:00.047508 6515 raft_consensus.cc:3055] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:00.047802 6515 raft_consensus.cc:740] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:48:00.045567 7000 consensus_queue.cc:1048] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 19:48:00.048741 6515 consensus_queue.cc:260] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:48:00.050935 6515 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:00.055341 6743 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 3.
I20260807 19:48:00.056762 6573 leader_election.cc:304] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a5eb44ea4c497fb18c88506aa9e758, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:48:00.059908 6515 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e49dcb89a24c59b04f2cbdc32abea2 in term 3.
I20260807 19:48:00.060892 7012 raft_consensus.cc:2804] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:00.061386 7012 raft_consensus.cc:697] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 LEADER]: Becoming Leader. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:48:00.062314 7012 consensus_queue.cc:237] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:48:00.071475 6356 catalog_manager.cc:5697] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 reported cstate change: term changed from 2 to 3, leader changed from 559e9c50afd348178dc1725870a64588 (127.6.42.193) to f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:00.163067 6515 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:00.164425 7012 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:00.173166 6742 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:00.175032 7012 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:00.233290 6515 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:00.234591 7012 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:00.243845 6742 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:00.246474 7046 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:48:00.372053 6742 raft_consensus.cc:1275] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:00.373389 7046 consensus_queue.cc:1048] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:00.382021 6515 raft_consensus.cc:1275] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:00.383805 7010 consensus_queue.cc:1048] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:00.445818 6742 raft_consensus.cc:1275] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:00.447031 7010 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:00.457684 6515 raft_consensus.cc:1275] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Refusing update from remote peer f4e49dcb89a24c59b04f2cbdc32abea2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:00.459894 7010 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:02.547796 6315 auto_leader_rebalancer-test.cc:436] The leader distribution is
I20260807 19:48:02.548714 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:02.555992 6627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:36108
I20260807 19:48:02.556681 6627 raft_consensus.cc:606] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Received request to transfer leadership to TS b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:48:02.559775 6627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:36108
I20260807 19:48:02.560319 6627 raft_consensus.cc:606] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 LEADER]: Received request to transfer leadership to TS 559e9c50afd348178dc1725870a64588
I20260807 19:48:02.562862 6627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:36108
I20260807 19:48:02.563427 6627 raft_consensus.cc:606] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Received request to transfer leadership to TS b5a5eb44ea4c497fb18c88506aa9e758
I20260807 19:48:02.565817 6627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:36108
I20260807 19:48:02.566306 6627 raft_consensus.cc:606] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Received request to transfer leadership to TS 559e9c50afd348178dc1725870a64588
I20260807 19:48:02.568698 6627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e"
dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
mode: GRACEFUL
new_leader_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:36108
I20260807 19:48:02.569195 6627 raft_consensus.cc:606] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Received request to transfer leadership to TS 559e9c50afd348178dc1725870a64588
I20260807 19:48:02.570199 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260807 19:48:02.570616 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:02.575014 7059 raft_consensus.cc:993] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:48:02.575543 7010 raft_consensus.cc:1081] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Signalling peer b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:48:02.577102 6743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:53338
I20260807 19:48:02.577606 6743 raft_consensus.cc:493] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:02.577921 6743 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:02.583112 6743 raft_consensus.cc:515] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.586089 6743 leader_election.cc:290] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 3 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:48:02.586517 6515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:48:02.586695 6627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1685c500eccb455f8707900cf9763d5c" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:48:02.587313 6515 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:02.587384 6627 raft_consensus.cc:3055] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:02.587792 6627 raft_consensus.cc:740] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:48:02.588721 6627 consensus_queue.cc:260] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.590167 6627 raft_consensus.cc:3060] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:02.592643 6515 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 3.
I20260807 19:48:02.593982 6689 leader_election.cc:304] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:48:02.594980 7060 raft_consensus.cc:2804] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:02.596056 7060 raft_consensus.cc:697] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:48:02.596218 6627 raft_consensus.cc:2468] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 3.
I20260807 19:48:02.596993 7060 consensus_queue.cc:237] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.605026 6358 catalog_manager.cc:5697] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 2 to 3, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:02.680191 7059 raft_consensus.cc:993] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:48:02.680696 7010 raft_consensus.cc:1081] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Signalling peer 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:48:02.682183 6515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41"
dest_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:39056
I20260807 19:48:02.682626 6515 raft_consensus.cc:493] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:02.682893 6515 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:48:02.687517 6515 raft_consensus.cc:515] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.689601 6515 leader_election.cc:290] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 5 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:48:02.690569 6627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:48:02.690757 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df6aaeeb7b9b46abbfa682af10a1bf41" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:48:02.691111 6627 raft_consensus.cc:3055] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:48:02.691422 6743 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:48:02.691435 6627 raft_consensus.cc:740] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:48:02.692267 6627 consensus_queue.cc:260] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.693303 6627 raft_consensus.cc:3060] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:48:02.695887 6743 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 5.
I20260807 19:48:02.697199 6464 leader_election.cc:304] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:48:02.698722 6627 raft_consensus.cc:2468] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 5.
I20260807 19:48:02.698980 7068 raft_consensus.cc:2804] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:48:02.700425 7068 raft_consensus.cc:697] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 5 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:48:02.701169 7068 consensus_queue.cc:237] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.708413 6358 catalog_manager.cc:5697] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 4 to 5, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 5 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:02.951361 7059 raft_consensus.cc:993] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:48:02.951881 7010 raft_consensus.cc:1081] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Signalling peer b5a5eb44ea4c497fb18c88506aa9e758 to start an election
I20260807 19:48:02.953495 6743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516"
dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
from {username='slave'} at 127.0.0.1:53338
I20260807 19:48:02.953963 6743 raft_consensus.cc:493] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:02.954227 6743 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:48:02.955595 7010 raft_consensus.cc:993] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:48:02.956192 7059 raft_consensus.cc:1081] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Signalling peer 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:48:02.957928 6515 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e"
dest_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:39056
I20260807 19:48:02.958446 6515 raft_consensus.cc:493] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:02.958778 6515 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:02.960043 6743 raft_consensus.cc:515] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.962416 6743 leader_election.cc:290] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 5 election: Requested vote from peers 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611), f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551)
I20260807 19:48:02.963356 6516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "559e9c50afd348178dc1725870a64588"
I20260807 19:48:02.963958 6516 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:48:02.964411 6627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbe73bcb11b04d5d9dfea85d210f5516" candidate_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:48:02.965000 6627 raft_consensus.cc:3055] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:48:02.965302 6627 raft_consensus.cc:740] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:48:02.965361 6515 raft_consensus.cc:515] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.966213 6627 consensus_queue.cc:260] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.967674 6627 raft_consensus.cc:3060] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:48:02.967965 6515 leader_election.cc:290] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 3 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:48:02.969506 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:48:02.970112 6629 raft_consensus.cc:3055] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:02.970407 6629 raft_consensus.cc:740] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:48:02.971335 6516 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 5.
I20260807 19:48:02.971254 6629 consensus_queue.cc:260] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.972524 6629 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:02.973982 6689 leader_election.cc:304] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:48:02.974174 6627 raft_consensus.cc:2468] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5a5eb44ea4c497fb18c88506aa9e758 in term 5.
I20260807 19:48:02.975118 7060 raft_consensus.cc:2804] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:48:02.975113 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc063e9b3c64636b307a0f06f78494e" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:48:02.975822 7060 raft_consensus.cc:697] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 LEADER]: Becoming Leader. State: Replica: b5a5eb44ea4c497fb18c88506aa9e758, State: Running, Role: LEADER
I20260807 19:48:02.975894 6743 raft_consensus.cc:3060] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:02.977030 7060 consensus_queue.cc:237] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.979755 6629 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 3.
I20260807 19:48:02.981256 6465 leader_election.cc:304] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, f4e49dcb89a24c59b04f2cbdc32abea2; no voters:
I20260807 19:48:02.982173 7068 raft_consensus.cc:2804] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:02.982702 7068 raft_consensus.cc:697] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 3 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:48:02.983335 6743 raft_consensus.cc:2468] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 3.
I20260807 19:48:02.983525 7068 consensus_queue.cc:237] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } }
I20260807 19:48:02.987138 6358 catalog_manager.cc:5697] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 reported cstate change: term changed from 4 to 5, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:02.991477 6356 catalog_manager.cc:5697] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 2 to 3, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 3 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:03.028741 7010 raft_consensus.cc:993] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: : Instructing follower 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:48:03.029209 7059 raft_consensus.cc:1081] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 LEADER]: Signalling peer 559e9c50afd348178dc1725870a64588 to start an election
I20260807 19:48:03.030606 6516 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786"
dest_uuid: "559e9c50afd348178dc1725870a64588"
from {username='slave'} at 127.0.0.1:39056
I20260807 19:48:03.031064 6516 raft_consensus.cc:493] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:03.031323 6516 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:03.035737 6516 raft_consensus.cc:515] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:48:03.037457 6516 leader_election.cc:290] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 4 election: Requested vote from peers f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551), b5a5eb44ea4c497fb18c88506aa9e758 (127.6.42.195:36983)
I20260807 19:48:03.038597 6743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b5a5eb44ea4c497fb18c88506aa9e758"
I20260807 19:48:03.038844 6629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5916701f63fe4f61ade4abdd26f16786" candidate_uuid: "559e9c50afd348178dc1725870a64588" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2"
I20260807 19:48:03.039220 6743 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:03.039439 6629 raft_consensus.cc:3055] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:48:03.039762 6629 raft_consensus.cc:740] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f4e49dcb89a24c59b04f2cbdc32abea2, State: Running, Role: LEADER
I20260807 19:48:03.040514 6629 consensus_queue.cc:260] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:48:03.041499 6629 raft_consensus.cc:3060] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:03.044572 6743 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 4.
I20260807 19:48:03.045668 6464 leader_election.cc:304] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 559e9c50afd348178dc1725870a64588, b5a5eb44ea4c497fb18c88506aa9e758; no voters:
I20260807 19:48:03.046351 7068 raft_consensus.cc:2804] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:48:03.046710 6629 raft_consensus.cc:2468] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 559e9c50afd348178dc1725870a64588 in term 4.
I20260807 19:48:03.046764 7068 raft_consensus.cc:697] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 4 LEADER]: Becoming Leader. State: Replica: 559e9c50afd348178dc1725870a64588, State: Running, Role: LEADER
I20260807 19:48:03.047781 7068 consensus_queue.cc:237] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } }
I20260807 19:48:03.055433 6356 catalog_manager.cc:5697] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 reported cstate change: term changed from 3 to 4, leader changed from f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194) to 559e9c50afd348178dc1725870a64588 (127.6.42.193). New cstate: current_term: 4 leader_uuid: "559e9c50afd348178dc1725870a64588" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:03.093675 6629 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:48:03.095116 7068 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:48:03.106635 6743 raft_consensus.cc:1275] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:48:03.108377 7068 consensus_queue.cc:1048] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:48:03.185169 6629 raft_consensus.cc:1275] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:03.186504 7060 consensus_queue.cc:1048] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:03.196163 6516 raft_consensus.cc:1275] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:03.197799 7061 consensus_queue.cc:1048] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:48:03.465161 6743 raft_consensus.cc:1275] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:03.466367 7068 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:03.475425 6629 raft_consensus.cc:1275] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:03.477860 7069 consensus_queue.cc:1048] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:48:03.487381 6629 raft_consensus.cc:1275] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:03.488608 7069 consensus_queue.cc:1048] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:03.499696 6743 raft_consensus.cc:1275] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Refusing update from remote peer 559e9c50afd348178dc1725870a64588: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:03.501269 7069 consensus_queue.cc:1048] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a5eb44ea4c497fb18c88506aa9e758" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:03.568802 6629 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:48:03.570093 7088 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4e49dcb89a24c59b04f2cbdc32abea2" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 39551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:48:03.571862 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:03.576048 6516 raft_consensus.cc:1275] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Refusing update from remote peer b5a5eb44ea4c497fb18c88506aa9e758: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:48:03.577744 7060 consensus_queue.cc:1048] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "559e9c50afd348178dc1725870a64588" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:48:03.581543 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:03.582731 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:04.584046 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:04.590052 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:04.590518 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:05.591704 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:05.596913 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:05.597556 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:06.598912 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:06.605717 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:06.606331 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:07.607909 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:07.614848 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:07.615379 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:08.616560 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:08.622156 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:08.622602 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:09.623587 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:09.628677 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:09.629123 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:10.673972 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:10.679410 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:10.679879 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
W20260807 19:48:11.088797 6546 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6060 after lost signal to thread 6318
W20260807 19:48:11.089776 6546 debug-util.cc:398] Leaking SignalData structure 0x7b08000e6740 after lost signal to thread 6414
W20260807 19:48:11.090610 6546 debug-util.cc:398] Leaking SignalData structure 0x7b080018df80 after lost signal to thread 6550
W20260807 19:48:11.091398 6546 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7d60 after lost signal to thread 6663
W20260807 19:48:11.092127 6546 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4920 after lost signal to thread 6783
I20260807 19:48:11.681088 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:11.686580 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:11.687083 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:12.725991 6315 auto_leader_rebalancer-test.cc:448] The leader distribution is
I20260807 19:48:12.726737 6315 tablet_server.cc:179] TabletServer@127.6.42.193:0 shutting down...
W20260807 19:48:12.754528 6689 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260807 19:48:12.770612 6689 consensus_peers.cc:597] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 -> Peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Couldn't send request to peer 559e9c50afd348178dc1725870a64588. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:12.774042 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:12.775017 6315 tablet_replica.cc:333] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.775884 6315 raft_consensus.cc:2243] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 3 LEADER]: Raft consensus shutting down.
I20260807 19:48:12.777093 6315 raft_consensus.cc:2272] T edc063e9b3c64636b307a0f06f78494e P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.780131 6315 tablet_replica.cc:333] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.780725 6315 raft_consensus.cc:2243] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 5 LEADER]: Raft consensus shutting down.
I20260807 19:48:12.781629 6315 raft_consensus.cc:2272] T df6aaeeb7b9b46abbfa682af10a1bf41 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.783595 6315 tablet_replica.cc:333] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.784101 6315 raft_consensus.cc:2243] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:12.784621 6315 raft_consensus.cc:2272] T b8b716aaad814bc9ba10aeaed17309ad P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.786661 6315 tablet_replica.cc:333] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.787211 6315 raft_consensus.cc:2243] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:12.787715 6315 raft_consensus.cc:2272] T 1685c500eccb455f8707900cf9763d5c P 559e9c50afd348178dc1725870a64588 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.789789 6315 tablet_replica.cc:333] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.790275 6315 raft_consensus.cc:2243] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:12.790753 6315 raft_consensus.cc:2272] T f68b6cd9b52544508dd3350010f8be02 P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.792692 6315 tablet_replica.cc:333] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.793200 6315 raft_consensus.cc:2243] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:12.793651 6315 raft_consensus.cc:2272] T f66e312e1c23422aa03e3d63485f351c P 559e9c50afd348178dc1725870a64588 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.795585 6315 tablet_replica.cc:333] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.796039 6315 raft_consensus.cc:2243] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:12.796557 6315 raft_consensus.cc:2272] T cbe73bcb11b04d5d9dfea85d210f5516 P 559e9c50afd348178dc1725870a64588 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.798583 6315 tablet_replica.cc:333] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.799198 6315 raft_consensus.cc:2243] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 4 LEADER]: Raft consensus shutting down.
I20260807 19:48:12.800333 6315 raft_consensus.cc:2272] T 5916701f63fe4f61ade4abdd26f16786 P 559e9c50afd348178dc1725870a64588 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:12.802191 6315 tablet_replica.cc:333] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588: stopping tablet replica
I20260807 19:48:12.802645 6315 raft_consensus.cc:2243] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:12.803035 6315 raft_consensus.cc:2272] T 79c0e6d318974614931a9d66127a0fd1 P 559e9c50afd348178dc1725870a64588 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 19:48:12.826973 6315 connection.cc:466] server connection from 127.0.0.1:39056 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39056 (request call id 238) could send its response
W20260807 19:48:12.830617 6577 consensus_peers.cc:597] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 -> Peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Couldn't send request to peer 559e9c50afd348178dc1725870a64588. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:34611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 19:48:12.968012 6689 consensus_peers.cc:597] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 -> Peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Couldn't send request to peer 559e9c50afd348178dc1725870a64588. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:34611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 19:48:13.053985 6689 consensus_peers.cc:597] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 -> Peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Couldn't send request to peer 559e9c50afd348178dc1725870a64588. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:34611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 19:48:13.141790 6577 consensus_peers.cc:597] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 -> Peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Couldn't send request to peer 559e9c50afd348178dc1725870a64588. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:34611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260807 19:48:13.221155 6577 consensus_peers.cc:597] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 -> Peer 559e9c50afd348178dc1725870a64588 (127.6.42.193:34611): Couldn't send request to peer 559e9c50afd348178dc1725870a64588. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:34611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:13.547137 6315 tablet_server.cc:196] TabletServer@127.6.42.193:0 shutdown complete.
I20260807 19:48:13.570436 6315 tablet_server.cc:179] TabletServer@127.6.42.194:0 shutting down...
W20260807 19:48:13.596669 6688 consensus_peers.cc:597] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 -> Peer f4e49dcb89a24c59b04f2cbdc32abea2 (127.6.42.194:39551): Couldn't send request to peer f4e49dcb89a24c59b04f2cbdc32abea2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:13.603477 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:13.604298 6315 tablet_replica.cc:333] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.605031 6315 raft_consensus.cc:2243] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.605585 6315 raft_consensus.cc:2272] T df6aaeeb7b9b46abbfa682af10a1bf41 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.607473 6315 tablet_replica.cc:333] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.608001 6315 raft_consensus.cc:2243] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.608495 6315 raft_consensus.cc:2272] T cbe73bcb11b04d5d9dfea85d210f5516 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.610282 6315 tablet_replica.cc:333] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.610816 6315 raft_consensus.cc:2243] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.611651 6315 raft_consensus.cc:2272] T f68b6cd9b52544508dd3350010f8be02 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.613544 6315 tablet_replica.cc:333] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.614039 6315 raft_consensus.cc:2243] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.614814 6315 raft_consensus.cc:2272] T f66e312e1c23422aa03e3d63485f351c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.616600 6315 tablet_replica.cc:333] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.617084 6315 raft_consensus.cc:2243] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.617517 6315 raft_consensus.cc:2272] T edc063e9b3c64636b307a0f06f78494e P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.619266 6315 tablet_replica.cc:333] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.619714 6315 raft_consensus.cc:2243] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.620132 6315 raft_consensus.cc:2272] T 1685c500eccb455f8707900cf9763d5c P f4e49dcb89a24c59b04f2cbdc32abea2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.621783 6315 tablet_replica.cc:333] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.622227 6315 raft_consensus.cc:2243] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.622576 6315 raft_consensus.cc:2272] T b8b716aaad814bc9ba10aeaed17309ad P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.624171 6315 tablet_replica.cc:333] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.624653 6315 raft_consensus.cc:2243] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.625378 6315 raft_consensus.cc:2272] T 79c0e6d318974614931a9d66127a0fd1 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.627003 6315 tablet_replica.cc:333] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2: stopping tablet replica
I20260807 19:48:13.627444 6315 raft_consensus.cc:2243] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.627844 6315 raft_consensus.cc:2272] T 5916701f63fe4f61ade4abdd26f16786 P f4e49dcb89a24c59b04f2cbdc32abea2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.659793 6315 tablet_server.cc:196] TabletServer@127.6.42.194:0 shutdown complete.
I20260807 19:48:13.682152 6315 tablet_server.cc:179] TabletServer@127.6.42.195:0 shutting down...
I20260807 19:48:13.709717 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:13.710578 6315 tablet_replica.cc:333] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.711386 6315 raft_consensus.cc:2243] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.711951 6315 raft_consensus.cc:2272] T df6aaeeb7b9b46abbfa682af10a1bf41 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.714145 6315 tablet_replica.cc:333] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.714721 6315 raft_consensus.cc:2243] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.715241 6315 raft_consensus.cc:2272] T edc063e9b3c64636b307a0f06f78494e P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.717180 6315 tablet_replica.cc:333] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.717742 6315 raft_consensus.cc:2243] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.718570 6315 raft_consensus.cc:2272] T cbe73bcb11b04d5d9dfea85d210f5516 P b5a5eb44ea4c497fb18c88506aa9e758 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.720324 6315 tablet_replica.cc:333] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.720932 6315 raft_consensus.cc:2243] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.721433 6315 raft_consensus.cc:2272] T 5916701f63fe4f61ade4abdd26f16786 P b5a5eb44ea4c497fb18c88506aa9e758 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.723188 6315 tablet_replica.cc:333] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.723753 6315 raft_consensus.cc:2243] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.724154 6315 raft_consensus.cc:2272] T 79c0e6d318974614931a9d66127a0fd1 P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.725947 6315 tablet_replica.cc:333] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.726497 6315 raft_consensus.cc:2243] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.727259 6315 raft_consensus.cc:2272] T 1685c500eccb455f8707900cf9763d5c P b5a5eb44ea4c497fb18c88506aa9e758 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.728911 6315 tablet_replica.cc:333] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.729501 6315 raft_consensus.cc:2243] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.730263 6315 raft_consensus.cc:2272] T b8b716aaad814bc9ba10aeaed17309ad P b5a5eb44ea4c497fb18c88506aa9e758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.732044 6315 tablet_replica.cc:333] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.732548 6315 raft_consensus.cc:2243] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.732937 6315 raft_consensus.cc:2272] T f66e312e1c23422aa03e3d63485f351c P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.734612 6315 tablet_replica.cc:333] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758: stopping tablet replica
I20260807 19:48:13.735101 6315 raft_consensus.cc:2243] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:13.735473 6315 raft_consensus.cc:2272] T f68b6cd9b52544508dd3350010f8be02 P b5a5eb44ea4c497fb18c88506aa9e758 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.765852 6315 tablet_server.cc:196] TabletServer@127.6.42.195:0 shutdown complete.
I20260807 19:48:13.787046 6315 master.cc:562] Master@127.6.42.254:35443 shutting down...
I20260807 19:48:13.997574 6315 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:13.998227 6315 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:13.998541 6315 tablet_replica.cc:333] T 00000000000000000000000000000000 P 97201b96e55442659104305ba5f1eac0: stopping tablet replica
I20260807 19:48:14.019232 6315 master.cc:584] Master@127.6.42.254:35443 shutdown complete.
559e9c50afd348178dc1725870a64588 4
b5a5eb44ea4c497fb18c88506aa9e758 1
f4e49dcb89a24c59b04f2cbdc32abea2 4
559e9c50afd348178dc1725870a64588 3
b5a5eb44ea4c497fb18c88506aa9e758 3
f4e49dcb89a24c59b04f2cbdc32abea2 3
559e9c50afd348178dc1725870a64588 0
b5a5eb44ea4c497fb18c88506aa9e758 1
f4e49dcb89a24c59b04f2cbdc32abea2 8
559e9c50afd348178dc1725870a64588 3
b5a5eb44ea4c497fb18c88506aa9e758 3
f4e49dcb89a24c59b04f2cbdc32abea2 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (32166 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260807 19:48:14.065338 6315 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.42.254:40889
I20260807 19:48:14.066859 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:14.074119 7136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:14.074365 6315 server_base.cc:1061] running on GCE node
W20260807 19:48:14.074134 7141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:14.074131 7137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:14.075968 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:14.076185 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:14.076380 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132094076337 us; error 0 us; skew 500 ppm
I20260807 19:48:14.076908 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:14.079667 6315 webserver.cc:533] Webserver started at http://127.6.42.254:33227/ using document root <none> and password file <none>
I20260807 19:48:14.080184 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:14.080430 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:14.080716 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:14.081982 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/master-0-root/instance:
uuid: "d40f6e78a0294925a132c2514b007686"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.087103 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:48:14.091089 7146 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.091933 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:48:14.092260 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/master-0-root
uuid: "d40f6e78a0294925a132c2514b007686"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.092592 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:14.111678 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:14.113134 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:14.157752 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.254:40889
I20260807 19:48:14.157922 7226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.254:40889 every 8 connection(s)
I20260807 19:48:14.162159 7227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.172560 7227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686: Bootstrap starting.
I20260807 19:48:14.177615 7227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:14.183280 7227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686: No bootstrap required, opened a new log
I20260807 19:48:14.186336 7227 raft_consensus.cc:359] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d40f6e78a0294925a132c2514b007686" member_type: VOTER }
I20260807 19:48:14.187009 7227 raft_consensus.cc:385] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:14.187294 7227 raft_consensus.cc:740] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40f6e78a0294925a132c2514b007686, State: Initialized, Role: FOLLOWER
I20260807 19:48:14.187995 7227 consensus_queue.cc:260] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d40f6e78a0294925a132c2514b007686" member_type: VOTER }
I20260807 19:48:14.188555 7227 raft_consensus.cc:399] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:48:14.188836 7227 raft_consensus.cc:493] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:48:14.189244 7227 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:14.193754 7227 raft_consensus.cc:515] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d40f6e78a0294925a132c2514b007686" member_type: VOTER }
I20260807 19:48:14.194442 7227 leader_election.cc:304] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d40f6e78a0294925a132c2514b007686; no voters:
I20260807 19:48:14.195710 7227 leader_election.cc:290] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:48:14.196125 7232 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:14.197530 7232 raft_consensus.cc:697] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 1 LEADER]: Becoming Leader. State: Replica: d40f6e78a0294925a132c2514b007686, State: Running, Role: LEADER
I20260807 19:48:14.198329 7232 consensus_queue.cc:237] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d40f6e78a0294925a132c2514b007686" member_type: VOTER }
I20260807 19:48:14.198897 7227 sys_catalog.cc:565] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:48:14.203410 7234 sys_catalog.cc:455] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d40f6e78a0294925a132c2514b007686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d40f6e78a0294925a132c2514b007686" member_type: VOTER } }
I20260807 19:48:14.204386 7234 sys_catalog.cc:458] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:14.205482 7239 sys_catalog.cc:455] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d40f6e78a0294925a132c2514b007686. Latest consensus state: current_term: 1 leader_uuid: "d40f6e78a0294925a132c2514b007686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d40f6e78a0294925a132c2514b007686" member_type: VOTER } }
I20260807 19:48:14.206254 7239 sys_catalog.cc:458] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:14.207387 7249 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 19:48:14.213924 7249 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 19:48:14.216225 6315 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 19:48:14.223846 7249 catalog_manager.cc:1383] Generated new cluster ID: f60c94a4b3be483f9458f280ac2ac2de
I20260807 19:48:14.224256 7249 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 19:48:14.238988 7249 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 19:48:14.240410 7249 catalog_manager.cc:1540] Loading token signing keys...
I20260807 19:48:14.254026 7249 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686: Generated new TSK 0
I20260807 19:48:14.254695 7249 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 19:48:14.284458 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:14.291085 7264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:14.291632 7265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:14.294373 7269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:14.295197 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:14.296192 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:14.296449 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:14.296586 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132094296575 us; error 0 us; skew 500 ppm
I20260807 19:48:14.297065 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:14.299556 6315 webserver.cc:533] Webserver started at http://127.6.42.193:41493/ using document root <none> and password file <none>
I20260807 19:48:14.300038 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:14.300210 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:14.300550 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:14.301725 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-0-root/instance:
uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.306691 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260807 19:48:14.310454 7282 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.311270 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:48:14.311589 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-0-root
uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.311885 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:14.324270 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:14.325845 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:14.327560 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:14.330343 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:14.330606 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.330878 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:14.331053 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.384168 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.193:33987
I20260807 19:48:14.384279 7399 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.193:33987 every 8 connection(s)
I20260807 19:48:14.389495 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:14.397560 7409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:14.401055 7410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:14.401697 6315 server_base.cc:1061] running on GCE node
W20260807 19:48:14.401930 7413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:14.403056 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:14.403329 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:14.403498 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132094403480 us; error 0 us; skew 500 ppm
I20260807 19:48:14.404029 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:14.407425 6315 webserver.cc:533] Webserver started at http://127.6.42.194:45733/ using document root <none> and password file <none>
I20260807 19:48:14.408028 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:14.408427 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:14.408520 7404 heartbeater.cc:344] Connected to a master server at 127.6.42.254:40889
I20260807 19:48:14.408802 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:14.408963 7404 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:14.409926 7404 heartbeater.cc:507] Master 127.6.42.254:40889 requested a full tablet report, sending...
I20260807 19:48:14.410553 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-1-root/instance:
uuid: "7896cb60487144c19ba23a6e1eb91c84"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.412839 7177 ts_manager.cc:194] Registered new tserver with Master: 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987)
I20260807 19:48:14.414918 7177 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45980
I20260807 19:48:14.416620 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 19:48:14.420822 7419 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.421764 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:48:14.422119 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-1-root
uuid: "7896cb60487144c19ba23a6e1eb91c84"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.422420 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:14.440639 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:14.442027 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:14.443706 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:14.446144 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:14.446363 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.446596 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:14.446748 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.493470 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.194:32879
I20260807 19:48:14.493585 7525 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.194:32879 every 8 connection(s)
I20260807 19:48:14.498626 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:14.508726 7533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:14.508908 7532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:14.510789 7535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:14.512130 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:14.513687 7527 heartbeater.cc:344] Connected to a master server at 127.6.42.254:40889
I20260807 19:48:14.513747 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260807 19:48:14.514127 7527 heartbeater.cc:461] Registering TS with master...
W20260807 19:48:14.514124 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:14.514499 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132094514482 us; error 0 us; skew 500 ppm
I20260807 19:48:14.515069 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:14.515018 7527 heartbeater.cc:507] Master 127.6.42.254:40889 requested a full tablet report, sending...
I20260807 19:48:14.517535 7177 ts_manager.cc:194] Registered new tserver with Master: 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:14.517940 6315 webserver.cc:533] Webserver started at http://127.6.42.195:42309/ using document root <none> and password file <none>
I20260807 19:48:14.518458 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:14.518668 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:14.518918 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:14.519115 7177 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45992
I20260807 19:48:14.520277 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-2-root/instance:
uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.525556 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:48:14.529058 7544 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.529824 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 19:48:14.530128 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-2-root
uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
format_stamp: "Formatted at 2026-08-07 19:48:14 on dist-test-slave-lkx2"
I20260807 19:48:14.530442 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786132061667097-6315-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:14.551884 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:14.553221 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:14.555231 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:14.557554 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:14.557781 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.558043 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:14.558198 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:14.598912 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.195:45765
I20260807 19:48:14.599009 7641 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.195:45765 every 8 connection(s)
I20260807 19:48:14.613044 7642 heartbeater.cc:344] Connected to a master server at 127.6.42.254:40889
I20260807 19:48:14.613473 7642 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:14.614216 7642 heartbeater.cc:507] Master 127.6.42.254:40889 requested a full tablet report, sending...
I20260807 19:48:14.616182 7177 ts_manager.cc:194] Registered new tserver with Master: 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:14.617138 6315 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014956795s
I20260807 19:48:14.617974 7177 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46006
I20260807 19:48:14.647385 7177 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46008:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:48:14.650707 7177 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:48:14.734336 7467 tablet_service.cc:1511] Processing CreateTablet for tablet 28be4d1e918244e79b1651b304cd1d87 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260807 19:48:14.736124 7467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28be4d1e918244e79b1651b304cd1d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.741596 7466 tablet_service.cc:1511] Processing CreateTablet for tablet d9c1f998933e43bdad3ec94a95cbe2d0 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260807 19:48:14.743290 7466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c1f998933e43bdad3ec94a95cbe2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.745635 7463 tablet_service.cc:1511] Processing CreateTablet for tablet ab93a925d45749398c96b3f8faa369a5 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260807 19:48:14.746528 7589 tablet_service.cc:1511] Processing CreateTablet for tablet 28be4d1e918244e79b1651b304cd1d87 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260807 19:48:14.748037 7589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28be4d1e918244e79b1651b304cd1d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.748414 7459 tablet_service.cc:1511] Processing CreateTablet for tablet 62952c3d42b14218afacfcc9f43d71b9 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260807 19:48:14.748332 7460 tablet_service.cc:1511] Processing CreateTablet for tablet 0eeaffc15a61427f933c6f1adad12171 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260807 19:48:14.749589 7588 tablet_service.cc:1511] Processing CreateTablet for tablet d9c1f998933e43bdad3ec94a95cbe2d0 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260807 19:48:14.751360 7457 tablet_service.cc:1511] Processing CreateTablet for tablet 9ef375c9422c4abaa6ad923aa48a0383 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260807 19:48:14.751602 7456 tablet_service.cc:1511] Processing CreateTablet for tablet 28119b0c4c1d4cd0a114c7866b054148 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260807 19:48:14.752009 7587 tablet_service.cc:1511] Processing CreateTablet for tablet ab93a925d45749398c96b3f8faa369a5 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260807 19:48:14.754204 7454 tablet_service.cc:1511] Processing CreateTablet for tablet 1d113af7758c47c6840606de3d2b6cd1 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260807 19:48:14.754344 7584 tablet_service.cc:1511] Processing CreateTablet for tablet 0eeaffc15a61427f933c6f1adad12171 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260807 19:48:14.755635 7454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d113af7758c47c6840606de3d2b6cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.756649 7583 tablet_service.cc:1511] Processing CreateTablet for tablet 62952c3d42b14218afacfcc9f43d71b9 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260807 19:48:14.758858 7582 tablet_service.cc:1511] Processing CreateTablet for tablet 9ef375c9422c4abaa6ad923aa48a0383 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260807 19:48:14.759898 7456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28119b0c4c1d4cd0a114c7866b054148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.760690 7453 tablet_service.cc:1511] Processing CreateTablet for tablet 36f3d9f167ec4e74b3b19c51f07c8292 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260807 19:48:14.762075 7453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f3d9f167ec4e74b3b19c51f07c8292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.761974 7338 tablet_service.cc:1511] Processing CreateTablet for tablet 28be4d1e918244e79b1651b304cd1d87 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260807 19:48:14.763299 7338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28be4d1e918244e79b1651b304cd1d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.765774 7463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab93a925d45749398c96b3f8faa369a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.765604 7335 tablet_service.cc:1511] Processing CreateTablet for tablet d9c1f998933e43bdad3ec94a95cbe2d0 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260807 19:48:14.767012 7335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c1f998933e43bdad3ec94a95cbe2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.768756 7457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ef375c9422c4abaa6ad923aa48a0383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.769390 7582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ef375c9422c4abaa6ad923aa48a0383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.769600 7331 tablet_service.cc:1511] Processing CreateTablet for tablet ab93a925d45749398c96b3f8faa369a5 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260807 19:48:14.770937 7331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab93a925d45749398c96b3f8faa369a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.772221 7584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eeaffc15a61427f933c6f1adad12171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.772754 7459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62952c3d42b14218afacfcc9f43d71b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.773721 7330 tablet_service.cc:1511] Processing CreateTablet for tablet 0eeaffc15a61427f933c6f1adad12171 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260807 19:48:14.775167 7330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eeaffc15a61427f933c6f1adad12171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.776795 7460 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eeaffc15a61427f933c6f1adad12171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.776839 7583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62952c3d42b14218afacfcc9f43d71b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.777657 7329 tablet_service.cc:1511] Processing CreateTablet for tablet 62952c3d42b14218afacfcc9f43d71b9 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260807 19:48:14.779595 7587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab93a925d45749398c96b3f8faa369a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.780781 7588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9c1f998933e43bdad3ec94a95cbe2d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.781068 7325 tablet_service.cc:1511] Processing CreateTablet for tablet 36f3d9f167ec4e74b3b19c51f07c8292 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260807 19:48:14.783361 7578 tablet_service.cc:1511] Processing CreateTablet for tablet 28119b0c4c1d4cd0a114c7866b054148 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260807 19:48:14.784761 7578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28119b0c4c1d4cd0a114c7866b054148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.787436 7577 tablet_service.cc:1511] Processing CreateTablet for tablet 1d113af7758c47c6840606de3d2b6cd1 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260807 19:48:14.788897 7577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d113af7758c47c6840606de3d2b6cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.791175 7575 tablet_service.cc:1511] Processing CreateTablet for tablet 36f3d9f167ec4e74b3b19c51f07c8292 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260807 19:48:14.793624 7327 tablet_service.cc:1511] Processing CreateTablet for tablet 28119b0c4c1d4cd0a114c7866b054148 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260807 19:48:14.795735 7328 tablet_service.cc:1511] Processing CreateTablet for tablet 9ef375c9422c4abaa6ad923aa48a0383 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260807 19:48:14.797626 7326 tablet_service.cc:1511] Processing CreateTablet for tablet 1d113af7758c47c6840606de3d2b6cd1 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260807 19:48:14.804780 7575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f3d9f167ec4e74b3b19c51f07c8292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.814167 7325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36f3d9f167ec4e74b3b19c51f07c8292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.815685 7326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d113af7758c47c6840606de3d2b6cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.819053 7328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ef375c9422c4abaa6ad923aa48a0383. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.824677 7329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62952c3d42b14218afacfcc9f43d71b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.828317 7327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28119b0c4c1d4cd0a114c7866b054148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.840873 7324 tablet_service.cc:1511] Processing CreateTablet for tablet c6670d21774c4fb9bd04ec36b3de9cf3 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260807 19:48:14.842388 7324 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6670d21774c4fb9bd04ec36b3de9cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.873799 7452 tablet_service.cc:1511] Processing CreateTablet for tablet c6670d21774c4fb9bd04ec36b3de9cf3 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260807 19:48:14.875734 7452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6670d21774c4fb9bd04ec36b3de9cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.875524 7573 tablet_service.cc:1511] Processing CreateTablet for tablet c6670d21774c4fb9bd04ec36b3de9cf3 (DEFAULT_TABLE table=test-workload [id=e2e6e17b6509474b9e95397b52352cc7]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260807 19:48:14.876962 7573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6670d21774c4fb9bd04ec36b3de9cf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:14.967620 7675 tablet_bootstrap.cc:492] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:14.979977 7676 tablet_bootstrap.cc:492] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:14.982198 7677 tablet_bootstrap.cc:492] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:14.987852 7675 tablet_bootstrap.cc:654] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:14.989812 7676 tablet_bootstrap.cc:654] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.004271 7677 tablet_bootstrap.cc:654] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.004594 7675 tablet_bootstrap.cc:492] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.005120 7675 ts_tablet_manager.cc:1403] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.038s user 0.011s sys 0.003s
I20260807 19:48:15.006954 7676 tablet_bootstrap.cc:492] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.007503 7676 ts_tablet_manager.cc:1403] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.008s
I20260807 19:48:15.008314 7675 raft_consensus.cc:359] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.009114 7675 raft_consensus.cc:385] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.009438 7675 raft_consensus.cc:740] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.009593 7677 tablet_bootstrap.cc:492] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.010142 7677 ts_tablet_manager.cc:1403] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.006s
I20260807 19:48:15.010505 7675 consensus_queue.cc:260] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.010691 7676 raft_consensus.cc:359] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.011363 7676 raft_consensus.cc:385] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.011675 7676 raft_consensus.cc:740] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.012550 7677 raft_consensus.cc:359] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.013290 7677 raft_consensus.cc:385] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.012830 7676 consensus_queue.cc:260] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.013665 7677 raft_consensus.cc:740] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.013933 7527 heartbeater.cc:499] Master 127.6.42.254:40889 was elected leader, sending a full tablet report...
I20260807 19:48:15.014297 7675 ts_tablet_manager.cc:1434] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.009s user 0.002s sys 0.006s
I20260807 19:48:15.014442 7677 consensus_queue.cc:260] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.015576 7675 tablet_bootstrap.cc:492] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.023284 7675 tablet_bootstrap.cc:654] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.050527 7676 ts_tablet_manager.cc:1434] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.043s user 0.007s sys 0.035s
I20260807 19:48:15.050614 7642 heartbeater.cc:499] Master 127.6.42.254:40889 was elected leader, sending a full tablet report...
I20260807 19:48:15.050851 7677 ts_tablet_manager.cc:1434] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.040s user 0.016s sys 0.023s
I20260807 19:48:15.051740 7676 tablet_bootstrap.cc:492] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.052140 7404 heartbeater.cc:499] Master 127.6.42.254:40889 was elected leader, sending a full tablet report...
I20260807 19:48:15.053469 7675 tablet_bootstrap.cc:492] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.053969 7675 ts_tablet_manager.cc:1403] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.039s user 0.026s sys 0.009s
I20260807 19:48:15.057255 7677 tablet_bootstrap.cc:492] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.056854 7675 raft_consensus.cc:359] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.057817 7675 raft_consensus.cc:385] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.058295 7675 raft_consensus.cc:740] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.058840 7676 tablet_bootstrap.cc:654] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.059253 7675 consensus_queue.cc:260] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.061568 7675 ts_tablet_manager.cc:1434] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:48:15.062536 7675 tablet_bootstrap.cc:492] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.065886 7677 tablet_bootstrap.cc:654] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.069402 7675 tablet_bootstrap.cc:654] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.070189 7684 raft_consensus.cc:493] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.070845 7684 raft_consensus.cc:515] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.075964 7684 leader_election.cc:290] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.080917 7675 tablet_bootstrap.cc:492] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.081554 7675 ts_tablet_manager.cc:1403] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.004s
I20260807 19:48:15.086180 7677 tablet_bootstrap.cc:492] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.086740 7677 ts_tablet_manager.cc:1403] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.004s
I20260807 19:48:15.089805 7677 raft_consensus.cc:359] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.090037 7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.090613 7677 raft_consensus.cc:385] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.090967 7677 raft_consensus.cc:740] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
W20260807 19:48:15.091976 7546 leader_election.cc:343] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.091734 7677 consensus_queue.cc:260] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.093976 7677 ts_tablet_manager.cc:1434] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 19:48:15.094950 7677 tablet_bootstrap.cc:492] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.096586 7675 raft_consensus.cc:359] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.097366 7675 raft_consensus.cc:385] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.097694 7675 raft_consensus.cc:740] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.098879 7675 consensus_queue.cc:260] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.099969 7683 raft_consensus.cc:493] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.100474 7683 raft_consensus.cc:515] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.106089 7676 tablet_bootstrap.cc:492] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.106793 7676 ts_tablet_manager.cc:1403] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.055s user 0.025s sys 0.028s
I20260807 19:48:15.108479 7677 tablet_bootstrap.cc:654] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.110455 7676 raft_consensus.cc:359] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
W20260807 19:48:15.110930 7646 tablet.cc:2406] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:48:15.111443 7676 raft_consensus.cc:385] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.111969 7676 raft_consensus.cc:740] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.113030 7676 consensus_queue.cc:260] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.115974 7676 ts_tablet_manager.cc:1434] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20260807 19:48:15.117316 7676 tablet_bootstrap.cc:492] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.117616 7683 leader_election.cc:290] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.119414 7687 raft_consensus.cc:493] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.119947 7687 raft_consensus.cc:515] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.125468 7676 tablet_bootstrap.cc:654] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.126439 7687 leader_election.cc:290] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.129554 7684 raft_consensus.cc:493] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.130179 7684 raft_consensus.cc:515] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.134060 7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
W20260807 19:48:15.136209 7546 leader_election.cc:343] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.137641 7684 leader_election.cc:290] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.148499 7683 raft_consensus.cc:493] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.149192 7683 raft_consensus.cc:515] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.151693 7683 leader_election.cc:290] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.152940 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.152248 7478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.154953 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6670d21774c4fb9bd04ec36b3de9cf3" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
W20260807 19:48:15.155697 7547 leader_election.cc:343] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.156212 7547 leader_election.cc:304] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84
W20260807 19:48:15.156610 7287 leader_election.cc:343] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.157554 7684 raft_consensus.cc:2749] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:48:15.158937 7547 leader_election.cc:343] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.159389 7547 leader_election.cc:304] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:15.160492 7684 raft_consensus.cc:2749] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:15.164569 7676 tablet_bootstrap.cc:492] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.165256 7676 ts_tablet_manager.cc:1403] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.048s user 0.026s sys 0.014s
I20260807 19:48:15.165450 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6670d21774c4fb9bd04ec36b3de9cf3" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
W20260807 19:48:15.167244 7285 leader_election.cc:343] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.167732 7285 leader_election.cc:304] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:15.169031 7687 raft_consensus.cc:2749] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:15.168993 7676 raft_consensus.cc:359] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.177673 7676 raft_consensus.cc:385] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.178164 7676 raft_consensus.cc:740] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.179442 7676 consensus_queue.cc:260] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.182406 7676 ts_tablet_manager.cc:1434] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.017s user 0.002s sys 0.005s
I20260807 19:48:15.191294 7676 tablet_bootstrap.cc:492] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.199286 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.200140 7676 tablet_bootstrap.cc:654] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.200827 7607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.202394 7675 ts_tablet_manager.cc:1434] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.120s user 0.039s sys 0.032s
W20260807 19:48:15.202998 7422 leader_election.cc:343] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:15.203895 7422 leader_election.cc:343] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.205365 7675 tablet_bootstrap.cc:492] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
W20260807 19:48:15.211124 7252 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:15.212931 7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.213819 7676 tablet_bootstrap.cc:492] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.213081 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.214579 7676 ts_tablet_manager.cc:1403] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.023s user 0.015s sys 0.005s
I20260807 19:48:15.214766 7350 raft_consensus.cc:2468] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 0.
I20260807 19:48:15.215312 7675 tablet_bootstrap.cc:654] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:15.216307 7422 leader_election.cc:343] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.216848 7422 leader_election.cc:304] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7896cb60487144c19ba23a6e1eb91c84; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:15.218191 7422 leader_election.cc:304] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:15.219123 7697 raft_consensus.cc:2749] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:15.218669 7676 raft_consensus.cc:359] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.219906 7676 raft_consensus.cc:385] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.220324 7676 raft_consensus.cc:740] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.221295 7676 consensus_queue.cc:260] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.224181 7676 ts_tablet_manager.cc:1434] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:48:15.225365 7676 tablet_bootstrap.cc:492] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.225493 7697 raft_consensus.cc:2804] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.225973 7697 raft_consensus.cc:493] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.226322 7697 raft_consensus.cc:3060] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.231814 7675 tablet_bootstrap.cc:492] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.232501 7675 ts_tablet_manager.cc:1403] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.027s user 0.016s sys 0.004s
I20260807 19:48:15.233567 7677 tablet_bootstrap.cc:492] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.233564 7676 tablet_bootstrap.cc:654] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.234154 7677 ts_tablet_manager.cc:1403] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.139s user 0.029s sys 0.045s
I20260807 19:48:15.236081 7675 raft_consensus.cc:359] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.236882 7675 raft_consensus.cc:385] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.237200 7675 raft_consensus.cc:740] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.237205 7677 raft_consensus.cc:359] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.237948 7677 raft_consensus.cc:385] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.238198 7677 raft_consensus.cc:740] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.238230 7675 consensus_queue.cc:260] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.238811 7677 consensus_queue.cc:260] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.240443 7697 raft_consensus.cc:515] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.241017 7677 ts_tablet_manager.cc:1434] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:48:15.241997 7677 tablet_bootstrap.cc:492] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.242669 7697 leader_election.cc:290] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.244727 7675 ts_tablet_manager.cc:1434] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260807 19:48:15.245764 7675 tablet_bootstrap.cc:492] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.246142 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.247202 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
W20260807 19:48:15.247825 7422 leader_election.cc:343] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.249722 7677 tablet_bootstrap.cc:654] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.247774 7350 raft_consensus.cc:3060] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.252130 7675 tablet_bootstrap.cc:654] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.256829 7676 tablet_bootstrap.cc:492] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.257416 7676 ts_tablet_manager.cc:1403] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.000s
I20260807 19:48:15.260071 7350 raft_consensus.cc:2468] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 1.
I20260807 19:48:15.260658 7675 tablet_bootstrap.cc:492] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.261303 7675 ts_tablet_manager.cc:1403] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260807 19:48:15.260579 7676 raft_consensus.cc:359] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.261502 7422 leader_election.cc:304] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:15.261690 7676 raft_consensus.cc:385] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.262202 7676 raft_consensus.cc:740] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.262710 7697 raft_consensus.cc:2804] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.263192 7697 raft_consensus.cc:697] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Becoming Leader. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:15.263252 7676 consensus_queue.cc:260] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.264127 7697 consensus_queue.cc:237] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.264268 7675 raft_consensus.cc:359] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.265131 7675 raft_consensus.cc:385] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.265455 7675 raft_consensus.cc:740] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.266224 7675 consensus_queue.cc:260] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.273783 7676 ts_tablet_manager.cc:1434] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260807 19:48:15.281584 7676 tablet_bootstrap.cc:492] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.283578 7174 catalog_manager.cc:5697] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "7896cb60487144c19ba23a6e1eb91c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:15.288252 7676 tablet_bootstrap.cc:654] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.294586 7675 ts_tablet_manager.cc:1434] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.033s user 0.017s sys 0.015s
I20260807 19:48:15.295678 7675 tablet_bootstrap.cc:492] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.295940 7677 tablet_bootstrap.cc:492] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.296432 7677 ts_tablet_manager.cc:1403] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.055s user 0.024s sys 0.019s
I20260807 19:48:15.297591 7676 tablet_bootstrap.cc:492] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.298218 7676 ts_tablet_manager.cc:1403] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260807 19:48:15.300052 7677 raft_consensus.cc:359] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.300922 7677 raft_consensus.cc:385] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.301222 7677 raft_consensus.cc:740] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.302419 7675 tablet_bootstrap.cc:654] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.301968 7677 consensus_queue.cc:260] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.302076 7676 raft_consensus.cc:359] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.304600 7677 ts_tablet_manager.cc:1434] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:48:15.305634 7677 tablet_bootstrap.cc:492] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.313565 7677 tablet_bootstrap.cc:654] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.303077 7676 raft_consensus.cc:385] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.315105 7676 raft_consensus.cc:740] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.316162 7676 consensus_queue.cc:260] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.318845 7676 ts_tablet_manager.cc:1434] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260807 19:48:15.320371 7676 tablet_bootstrap.cc:492] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.322388 7675 tablet_bootstrap.cc:492] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.322924 7675 ts_tablet_manager.cc:1403] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.027s user 0.015s sys 0.008s
I20260807 19:48:15.326092 7675 raft_consensus.cc:359] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.326853 7675 raft_consensus.cc:385] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.327164 7675 raft_consensus.cc:740] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.328061 7676 tablet_bootstrap.cc:654] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.327987 7675 consensus_queue.cc:260] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.330678 7675 ts_tablet_manager.cc:1434] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:48:15.331789 7675 tablet_bootstrap.cc:492] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.338388 7675 tablet_bootstrap.cc:654] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.341625 7677 tablet_bootstrap.cc:492] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.342199 7677 ts_tablet_manager.cc:1403] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.021s
I20260807 19:48:15.344978 7676 tablet_bootstrap.cc:492] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.345595 7676 ts_tablet_manager.cc:1403] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.012s
I20260807 19:48:15.345351 7677 raft_consensus.cc:359] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.345818 7675 tablet_bootstrap.cc:492] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.346096 7677 raft_consensus.cc:385] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.346338 7675 ts_tablet_manager.cc:1403] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.001s
I20260807 19:48:15.346477 7677 raft_consensus.cc:740] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.347548 7677 consensus_queue.cc:260] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.349331 7676 raft_consensus.cc:359] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.350020 7677 ts_tablet_manager.cc:1434] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 19:48:15.350189 7676 raft_consensus.cc:385] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.350617 7676 raft_consensus.cc:740] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.351118 7677 tablet_bootstrap.cc:492] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.351617 7675 raft_consensus.cc:359] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.351570 7676 consensus_queue.cc:260] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.352512 7675 raft_consensus.cc:385] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.352849 7675 raft_consensus.cc:740] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.353714 7675 consensus_queue.cc:260] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.354636 7676 ts_tablet_manager.cc:1434] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.008s user 0.006s sys 0.002s
I20260807 19:48:15.355872 7676 tablet_bootstrap.cc:492] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.358465 7677 tablet_bootstrap.cc:654] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.355894 7675 ts_tablet_manager.cc:1434] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260807 19:48:15.359824 7675 tablet_bootstrap.cc:492] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.367383 7675 tablet_bootstrap.cc:654] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.370519 7677 tablet_bootstrap.cc:492] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.371070 7677 ts_tablet_manager.cc:1403] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.007s
I20260807 19:48:15.371858 7676 tablet_bootstrap.cc:654] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.374203 7677 raft_consensus.cc:359] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.374986 7677 raft_consensus.cc:385] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.375303 7677 raft_consensus.cc:740] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.376106 7677 consensus_queue.cc:260] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.378434 7677 ts_tablet_manager.cc:1434] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:48:15.379436 7677 tablet_bootstrap.cc:492] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.385754 7677 tablet_bootstrap.cc:654] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.392612 7684 raft_consensus.cc:493] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.393282 7684 raft_consensus.cc:515] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.397138 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.397665 7684 leader_election.cc:290] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.398378 7676 tablet_bootstrap.cc:492] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.398204 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.399070 7350 raft_consensus.cc:2393] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1aa4e6fd8a854e6ab3f2b073de50781e in current term 1: Already voted for candidate 7896cb60487144c19ba23a6e1eb91c84 in this term.
I20260807 19:48:15.399560 7676 ts_tablet_manager.cc:1403] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.044s user 0.016s sys 0.015s
I20260807 19:48:15.400291 7546 leader_election.cc:304] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:15.400827 7677 tablet_bootstrap.cc:492] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.401031 7684 raft_consensus.cc:3060] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.401333 7677 ts_tablet_manager.cc:1403] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.008s
I20260807 19:48:15.402069 7675 tablet_bootstrap.cc:492] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.402518 7675 ts_tablet_manager.cc:1403] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.043s user 0.023s sys 0.007s
I20260807 19:48:15.402407 7676 raft_consensus.cc:359] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.403103 7676 raft_consensus.cc:385] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.403394 7676 raft_consensus.cc:740] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.404081 7676 consensus_queue.cc:260] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.404255 7677 raft_consensus.cc:359] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.404932 7677 raft_consensus.cc:385] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.405243 7677 raft_consensus.cc:740] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.405350 7675 raft_consensus.cc:359] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.406028 7675 raft_consensus.cc:385] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.406292 7675 raft_consensus.cc:740] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.406214 7677 consensus_queue.cc:260] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.407178 7675 consensus_queue.cc:260] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.409487 7675 ts_tablet_manager.cc:1434] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:48:15.410458 7675 tablet_bootstrap.cc:492] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.414044 7677 ts_tablet_manager.cc:1434] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.012s user 0.005s sys 0.003s
I20260807 19:48:15.415220 7677 tablet_bootstrap.cc:492] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.415868 7675 tablet_bootstrap.cc:654] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.424870 7677 tablet_bootstrap.cc:654] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.427398 7676 ts_tablet_manager.cc:1434] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.027s user 0.019s sys 0.007s
I20260807 19:48:15.428138 7675 tablet_bootstrap.cc:492] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.428622 7675 ts_tablet_manager.cc:1403] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20260807 19:48:15.429662 7676 tablet_bootstrap.cc:492] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.431149 7675 raft_consensus.cc:359] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.431814 7675 raft_consensus.cc:385] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.432106 7675 raft_consensus.cc:740] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.433209 7675 consensus_queue.cc:260] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.435447 7675 ts_tablet_manager.cc:1434] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:48:15.438390 7684 raft_consensus.cc:2749] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:15.439188 7675 tablet_bootstrap.cc:492] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84: Bootstrap starting.
I20260807 19:48:15.437073 7676 tablet_bootstrap.cc:654] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.441573 7687 raft_consensus.cc:493] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.442143 7687 raft_consensus.cc:515] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.444373 7687 leader_election.cc:290] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.446173 7675 tablet_bootstrap.cc:654] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.447141 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.447881 7477 raft_consensus.cc:2468] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.449080 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.449775 7610 raft_consensus.cc:2468] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.451412 7285 leader_election.cc:304] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.453382 7687 raft_consensus.cc:2804] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.453819 7687 raft_consensus.cc:493] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.454160 7687 raft_consensus.cc:3060] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.455006 7684 raft_consensus.cc:493] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.455583 7684 raft_consensus.cc:515] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.457206 7737 raft_consensus.cc:493] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.458655 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.458864 7684 leader_election.cc:290] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.459771 7737 raft_consensus.cc:515] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
W20260807 19:48:15.460821 7546 leader_election.cc:343] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.461220 7675 tablet_bootstrap.cc:492] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84: No bootstrap required, opened a new log
I20260807 19:48:15.461786 7675 ts_tablet_manager.cc:1403] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84: Time spent bootstrapping tablet: real 0.023s user 0.013s sys 0.004s
I20260807 19:48:15.461827 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.462612 7477 raft_consensus.cc:2468] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 0.
I20260807 19:48:15.462970 7713 raft_consensus.cc:493] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.463438 7713 raft_consensus.cc:515] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.463989 7547 leader_election.cc:304] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:15.464452 7478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.465091 7478 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.464910 7675 raft_consensus.cc:359] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.465560 7675 raft_consensus.cc:385] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.465821 7675 raft_consensus.cc:740] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.466548 7287 leader_election.cc:304] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:15.466907 7675 consensus_queue.cc:260] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.469002 7675 ts_tablet_manager.cc:1434] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:48:15.470032 7737 leader_election.cc:290] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.470281 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.470774 7737 raft_consensus.cc:2804] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.470912 7610 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.474445 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f3d9f167ec4e74b3b19c51f07c8292" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.475077 7350 raft_consensus.cc:2468] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 0.
I20260807 19:48:15.471048 7713 leader_election.cc:290] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.468304 7684 raft_consensus.cc:2804] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.476735 7684 raft_consensus.cc:493] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.477111 7684 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.478209 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f3d9f167ec4e74b3b19c51f07c8292" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
W20260807 19:48:15.482110 7422 leader_election.cc:343] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.482956 7422 leader_election.cc:304] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:15.486043 7713 raft_consensus.cc:2804] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.486850 7713 raft_consensus.cc:493] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.477710 7737 raft_consensus.cc:493] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.487197 7713 raft_consensus.cc:3060] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.487967 7737 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.490998 7687 raft_consensus.cc:515] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.494750 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.495496 7610 raft_consensus.cc:3060] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.497996 7737 raft_consensus.cc:515] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.500083 7684 raft_consensus.cc:515] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.501756 7607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.504170 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:15.500252 7737 leader_election.cc:290] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.502449 7607 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.508690 7677 tablet_bootstrap.cc:492] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.509188 7677 ts_tablet_manager.cc:1403] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.094s user 0.027s sys 0.043s
I20260807 19:48:15.513080 7478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.513846 7478 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:15.516554 7546 leader_election.cc:343] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.518116 7677 raft_consensus.cc:359] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.506099 7687 leader_election.cc:290] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.518900 7677 raft_consensus.cc:385] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.519212 7677 raft_consensus.cc:740] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.520704 7610 raft_consensus.cc:2468] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.522145 7285 leader_election.cc:304] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.522297 7478 raft_consensus.cc:2468] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 1.
I20260807 19:48:15.523064 7687 raft_consensus.cc:2804] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.524148 7687 raft_consensus.cc:697] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:15.520020 7677 consensus_queue.cc:260] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.525091 7687 consensus_queue.cc:237] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.526859 7677 ts_tablet_manager.cc:1434] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260807 19:48:15.523926 7713 raft_consensus.cc:515] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.527876 7677 tablet_bootstrap.cc:492] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.528885 7547 leader_election.cc:304] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:15.529290 7713 leader_election.cc:290] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.529832 7684 leader_election.cc:290] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.530015 7732 raft_consensus.cc:2804] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election won for term 1
W20260807 19:48:15.532756 7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:32879, user_credentials={real_user=slave}} blocked reactor thread for 66293.2us
I20260807 19:48:15.535621 7607 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.535794 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f3d9f167ec4e74b3b19c51f07c8292" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:15.536521 7350 raft_consensus.cc:3060] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.536979 7285 leader_election.cc:304] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.538430 7732 raft_consensus.cc:697] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:15.538522 7738 raft_consensus.cc:2804] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.531972 7737 raft_consensus.cc:493] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.539742 7732 consensus_queue.cc:237] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.540623 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f3d9f167ec4e74b3b19c51f07c8292" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.539888 7738 raft_consensus.cc:697] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:15.541890 7737 raft_consensus.cc:515] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
W20260807 19:48:15.543375 7422 leader_election.cc:343] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.546523 7676 tablet_bootstrap.cc:492] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.546451 7478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.547104 7676 ts_tablet_manager.cc:1403] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.118s user 0.019s sys 0.023s
I20260807 19:48:15.547219 7478 raft_consensus.cc:3060] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.543752 7677 tablet_bootstrap.cc:654] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.549388 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.544824 7684 raft_consensus.cc:493] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.550089 7737 leader_election.cc:290] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.550333 7610 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.553603 7475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.551923 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.554298 7285 leader_election.cc:304] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.554849 7477 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.555527 7737 raft_consensus.cc:2804] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.556143 7737 raft_consensus.cc:493] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.556555 7737 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.558571 7676 raft_consensus.cc:359] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.559362 7676 raft_consensus.cc:385] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.559638 7676 raft_consensus.cc:740] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.560433 7676 consensus_queue.cc:260] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.554347 7475 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.552701 7684 raft_consensus.cc:515] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.566318 7684 leader_election.cc:290] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.567708 7475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.568228 7351 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.568454 7475 raft_consensus.cc:2468] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 0.
I20260807 19:48:15.569768 7547 leader_election.cc:304] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
W20260807 19:48:15.570637 7546 leader_election.cc:343] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.571743 7745 raft_consensus.cc:493] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.543701 7738 consensus_queue.cc:237] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.573837 7748 raft_consensus.cc:2804] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.574272 7748 raft_consensus.cc:493] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.574618 7748 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.573836 7745 raft_consensus.cc:515] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.577472 7737 raft_consensus.cc:515] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.580641 7737 leader_election.cc:290] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.582072 7745 leader_election.cc:290] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.582280 7475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.582813 7607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.583513 7607 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.583154 7475 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.582945 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.584731 7676 ts_tablet_manager.cc:1434] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.037s user 0.009s sys 0.001s
I20260807 19:48:15.586652 7676 tablet_bootstrap.cc:492] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e: Bootstrap starting.
I20260807 19:48:15.575227 7478 raft_consensus.cc:2468] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.571733 7350 raft_consensus.cc:2468] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 1.
I20260807 19:48:15.582363 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.589818 7474 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.590579 7477 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.586323 7610 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.594920 7422 leader_election.cc:304] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:15.596196 7713 raft_consensus.cc:2804] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.596745 7713 raft_consensus.cc:697] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Becoming Leader. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:15.597566 7713 consensus_queue.cc:237] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.588901 7287 leader_election.cc:304] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:15.604606 7607 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.604979 7745 raft_consensus.cc:2804] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.605412 7745 raft_consensus.cc:493] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.605775 7745 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.615756 7285 leader_election.cc:304] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.618650 7474 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.618281 7174 catalog_manager.cc:5697] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 1 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:15.622792 7677 tablet_bootstrap.cc:492] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.623317 7677 ts_tablet_manager.cc:1403] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.096s user 0.016s sys 0.010s
I20260807 19:48:15.620710 7177 catalog_manager.cc:5697] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 1 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:15.628684 7687 raft_consensus.cc:2804] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.631568 7676 tablet_bootstrap.cc:654] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.630888 7171 catalog_manager.cc:5697] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "7896cb60487144c19ba23a6e1eb91c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:15.633423 7687 raft_consensus.cc:697] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:15.634707 7687 consensus_queue.cc:237] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.636842 7677 raft_consensus.cc:359] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.637879 7677 raft_consensus.cc:385] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.638131 7677 raft_consensus.cc:740] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.639191 7677 consensus_queue.cc:260] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.642999 7745 raft_consensus.cc:515] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.647075 7745 leader_election.cc:290] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.648429 7607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.650309 7607 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.652988 7738 raft_consensus.cc:493] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.653254 7748 raft_consensus.cc:515] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.653602 7738 raft_consensus.cc:515] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.655714 7738 leader_election.cc:290] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.656740 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.656877 7478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.657166 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:15.657549 7478 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.657953 7477 raft_consensus.cc:2468] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.659319 7287 leader_election.cc:304] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:15.660010 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:15.661698 7677 ts_tablet_manager.cc:1434] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.038s user 0.009s sys 0.000s
I20260807 19:48:15.662794 7677 tablet_bootstrap.cc:492] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Bootstrap starting.
I20260807 19:48:15.664119 7738 raft_consensus.cc:2804] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.664466 7610 raft_consensus.cc:2468] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 0.
I20260807 19:48:15.664619 7738 raft_consensus.cc:493] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:48:15.665005 7546 leader_election.cc:343] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:15.665287 7738 raft_consensus.cc:3060] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.661677 7748 leader_election.cc:290] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.657582 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.674444 7676 tablet_bootstrap.cc:492] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e: No bootstrap required, opened a new log
I20260807 19:48:15.674851 7676 ts_tablet_manager.cc:1403] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent bootstrapping tablet: real 0.089s user 0.007s sys 0.009s
I20260807 19:48:15.677997 7676 raft_consensus.cc:359] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.678663 7676 raft_consensus.cc:385] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.678939 7676 raft_consensus.cc:740] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.672243 7474 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.679245 7677 tablet_bootstrap.cc:654] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:15.679613 7676 consensus_queue.cc:260] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.682039 7676 ts_tablet_manager.cc:1434] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260807 19:48:15.684274 7738 raft_consensus.cc:515] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.686484 7478 raft_consensus.cc:2468] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 1.
I20260807 19:48:15.687462 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.687224 7738 leader_election.cc:290] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:15.690759 7477 raft_consensus.cc:3060] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.696312 7607 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.696655 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:15.697371 7610 raft_consensus.cc:3060] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.700736 7285 leader_election.cc:304] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.701539 7738 raft_consensus.cc:2804] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.702363 7738 raft_consensus.cc:697] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:15.702672 7713 consensus_queue.cc:1048] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:15.703578 7738 consensus_queue.cc:237] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.707108 7677 tablet_bootstrap.cc:492] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e: No bootstrap required, opened a new log
I20260807 19:48:15.707625 7677 ts_tablet_manager.cc:1403] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent bootstrapping tablet: real 0.045s user 0.014s sys 0.010s
I20260807 19:48:15.711237 7610 raft_consensus.cc:2468] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.712560 7285 leader_election.cc:304] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.713582 7687 raft_consensus.cc:2804] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.714082 7687 raft_consensus.cc:697] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:15.714253 7547 leader_election.cc:304] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:15.714998 7687 consensus_queue.cc:237] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:15.715481 7748 raft_consensus.cc:2804] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.716037 7748 raft_consensus.cc:697] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:15.715942 7174 catalog_manager.cc:5697] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 1 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:15.717200 7748 consensus_queue.cc:237] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.718452 7174 catalog_manager.cc:5697] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 1 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:15.721110 7732 raft_consensus.cc:493] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.721638 7732 raft_consensus.cc:515] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.723538 7677 raft_consensus.cc:359] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.723917 7732 leader_election.cc:290] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.725782 7677 raft_consensus.cc:385] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:15.725827 7475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6670d21774c4fb9bd04ec36b3de9cf3" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.726154 7677 raft_consensus.cc:740] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Initialized, Role: FOLLOWER
I20260807 19:48:15.726466 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6670d21774c4fb9bd04ec36b3de9cf3" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.727113 7350 raft_consensus.cc:2468] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 0.
I20260807 19:48:15.728031 7677 consensus_queue.cc:260] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.728503 7546 leader_election.cc:304] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.726504 7475 raft_consensus.cc:2468] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 0.
I20260807 19:48:15.730753 7732 raft_consensus.cc:2804] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:15.731171 7732 raft_consensus.cc:493] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:15.731529 7732 raft_consensus.cc:3060] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.731786 7677 ts_tablet_manager.cc:1434] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e: Time spent starting tablet: real 0.024s user 0.004s sys 0.004s
I20260807 19:48:15.740067 7732 raft_consensus.cc:515] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.740739 7474 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.743036 7477 raft_consensus.cc:2468] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:15.743647 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6670d21774c4fb9bd04ec36b3de9cf3" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:15.744397 7350 raft_consensus.cc:3060] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.758065 7350 raft_consensus.cc:2468] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 1.
I20260807 19:48:15.759534 7546 leader_election.cc:304] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:15.765029 7748 raft_consensus.cc:2804] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:15.765604 7748 raft_consensus.cc:697] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:15.766537 7748 consensus_queue.cc:237] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.767711 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6670d21774c4fb9bd04ec36b3de9cf3" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:15.768476 7477 raft_consensus.cc:3060] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.777978 7732 leader_election.cc:290] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.777123 7174 catalog_manager.cc:5697] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 1 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:15.787410 7477 raft_consensus.cc:2468] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 1.
I20260807 19:48:15.790824 7697 consensus_queue.cc:1048] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:15.803470 7177 catalog_manager.cc:5697] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 1 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:15.805121 7177 catalog_manager.cc:5697] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 1 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:15.810001 7174 catalog_manager.cc:5697] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 1 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:15.859992 7748 raft_consensus.cc:493] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:15.860539 7748 raft_consensus.cc:515] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:15.862274 7748 leader_election.cc:290] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:15.863368 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f3d9f167ec4e74b3b19c51f07c8292" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" is_pre_election: true
I20260807 19:48:15.863372 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36f3d9f167ec4e74b3b19c51f07c8292" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:15.864408 7350 raft_consensus.cc:2393] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1aa4e6fd8a854e6ab3f2b073de50781e in current term 1: Already voted for candidate 7896cb60487144c19ba23a6e1eb91c84 in this term.
I20260807 19:48:15.865443 7546 leader_election.cc:304] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e; no voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:15.866114 7748 raft_consensus.cc:3060] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.872639 7748 raft_consensus.cc:2749] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:15.927129 7748 consensus_queue.cc:1048] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:15.941125 7350 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:15.950412 7684 consensus_queue.cc:1048] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:15.954203 7610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
mode: GRACEFUL
new_leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:40974
I20260807 19:48:15.954962 7610 raft_consensus.cc:606] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Received request to transfer leadership to TS 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:15.983258 7350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:48684
I20260807 19:48:15.983793 7350 raft_consensus.cc:606] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Received request to transfer leadership to TS 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:15.988873 7610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
mode: GRACEFUL
new_leader_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:40974
I20260807 19:48:15.989459 7610 raft_consensus.cc:606] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Received request to transfer leadership to TS 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:16.020980 7738 consensus_queue.cc:1048] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.031651 7737 consensus_queue.cc:1048] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.076395 7737 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.077406 7687 consensus_queue.cc:1048] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.084154 7737 raft_consensus.cc:993] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:16.085397 7745 raft_consensus.cc:1081] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Signalling peer 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:16.089382 7477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:43792
I20260807 19:48:16.090245 7737 consensus_queue.cc:1048] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.093647 7477 raft_consensus.cc:493] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:16.094175 7477 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.097633 7687 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:16.102401 7477 raft_consensus.cc:515] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.106875 7477 leader_election.cc:290] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:16.108068 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:16.108911 7350 raft_consensus.cc:3055] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:16.109336 7350 raft_consensus.cc:740] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:16.110430 7350 consensus_queue.cc:260] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.111768 7350 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.113245 7607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:16.121860 7350 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:16.124114 7422 leader_election.cc:304] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:16.125919 7697 raft_consensus.cc:2804] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:16.126747 7697 raft_consensus.cc:697] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Becoming Leader. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:16.127668 7697 consensus_queue.cc:237] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.131557 7687 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.138103 7687 consensus_queue.cc:1048] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.139451 7174 catalog_manager.cc:5697] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 reported cstate change: term changed from 1 to 2, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "7896cb60487144c19ba23a6e1eb91c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:16.149893 7737 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.163727 7745 consensus_queue.cc:1048] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.196635 7350 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:16.201859 7697 consensus_queue.cc:1048] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.204062 7684 consensus_queue.cc:1048] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.205876 7737 raft_consensus.cc:493] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:16.206427 7737 raft_consensus.cc:515] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.209357 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84" is_pre_election: true
I20260807 19:48:16.209985 7474 raft_consensus.cc:2468] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 1.
I20260807 19:48:16.211372 7287 leader_election.cc:304] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:16.213919 7737 leader_election.cc:290] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:16.214931 7745 raft_consensus.cc:2804] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:48:16.216076 7745 raft_consensus.cc:493] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1aa4e6fd8a854e6ab3f2b073de50781e)
I20260807 19:48:16.216537 7745 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.217974 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" is_pre_election: true
I20260807 19:48:16.222252 7732 raft_consensus.cc:993] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: : Instructing follower 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:16.222734 7732 raft_consensus.cc:1081] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Signalling peer 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:16.226331 7713 consensus_queue.cc:1048] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.226720 7745 raft_consensus.cc:515] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.230114 7350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:36398
I20260807 19:48:16.230726 7350 raft_consensus.cc:493] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:16.231122 7350 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:16.234297 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:16.234927 7474 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.243177 7474 raft_consensus.cc:2468] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 2.
I20260807 19:48:16.244098 7477 raft_consensus.cc:1240] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Rejecting Update request from peer 1aa4e6fd8a854e6ab3f2b073de50781e for earlier term 1. Current term is 2. Ops: []
I20260807 19:48:16.244685 7287 leader_election.cc:304] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:16.246529 7732 consensus_queue.cc:1059] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:16.247607 7748 raft_consensus.cc:3055] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:16.247969 7748 raft_consensus.cc:740] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:16.248898 7748 consensus_queue.cc:260] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.250029 7748 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.251775 7745 leader_election.cc:290] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 2 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:16.253345 7606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:16.258586 7350 raft_consensus.cc:515] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.259634 7737 raft_consensus.cc:2764] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260807 19:48:16.261015 7349 raft_consensus.cc:1240] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Rejecting Update request from peer 1aa4e6fd8a854e6ab3f2b073de50781e for earlier term 1. Current term is 3. Ops: []
I20260807 19:48:16.265476 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:16.265864 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9c1f998933e43bdad3ec94a95cbe2d0" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:16.266134 7474 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:16.267719 7350 leader_election.cc:290] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:16.269049 7606 raft_consensus.cc:2468] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 2.
I20260807 19:48:16.272428 7474 raft_consensus.cc:2468] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:16.273514 7287 leader_election.cc:304] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters: 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:16.274279 7737 raft_consensus.cc:2804] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:16.274691 7737 raft_consensus.cc:697] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:16.275492 7737 consensus_queue.cc:237] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.281802 7174 catalog_manager.cc:5697] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 1 to 3, leader changed from 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195) to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 3 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:16.320040 7748 consensus_queue.cc:1048] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.331787 7748 consensus_queue.cc:1048] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:16.346745 7684 raft_consensus.cc:993] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: : Instructing follower 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:16.347224 7748 raft_consensus.cc:1081] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Signalling peer 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:16.348703 7474 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:43786
I20260807 19:48:16.349154 7474 raft_consensus.cc:493] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:16.349421 7474 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.354321 7474 raft_consensus.cc:515] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.356168 7474 leader_election.cc:290] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:16.357132 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:16.357645 7350 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.357432 7606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:16.358407 7606 raft_consensus.cc:3055] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:16.358709 7606 raft_consensus.cc:740] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:16.359439 7606 consensus_queue.cc:260] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.360760 7606 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.363303 7350 raft_consensus.cc:2468] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:16.364636 7422 leader_election.cc:304] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:16.365526 7683 raft_consensus.cc:2804] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:16.365967 7683 raft_consensus.cc:697] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Becoming Leader. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:16.366333 7606 raft_consensus.cc:2468] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:16.366750 7683 consensus_queue.cc:237] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:16.373068 7174 catalog_manager.cc:5697] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 reported cstate change: term changed from 1 to 2, leader changed from 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195) to 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "7896cb60487144c19ba23a6e1eb91c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:16.584403 7606 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:16.589680 7606 raft_consensus.cc:1275] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:16.591147 7683 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:16.599804 7350 raft_consensus.cc:1275] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:16.601681 7683 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:16.792100 7610 raft_consensus.cc:1275] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:16.793399 7697 consensus_queue.cc:1048] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:16.803424 7350 raft_consensus.cc:1275] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:16.806094 7813 consensus_queue.cc:1048] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:16.813795 7610 raft_consensus.cc:3060] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:16.819861 7610 raft_consensus.cc:1275] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:48:16.821235 7737 consensus_queue.cc:1048] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:16.826959 7477 raft_consensus.cc:1275] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:48:16.829177 7745 consensus_queue.cc:1048] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:19.033156 6315 auto_leader_rebalancer-test.cc:473] The leader distribution is
I20260807 19:48:19.033946 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:19.052475 7350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:48692
I20260807 19:48:19.053041 7350 raft_consensus.cc:606] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Received request to transfer leadership to TS 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:19.055796 7350 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:48692
I20260807 19:48:19.056392 7350 raft_consensus.cc:606] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Received request to transfer leadership to TS 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:19.057436 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 19:48:19.057904 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:19.264473 7837 raft_consensus.cc:993] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:19.265070 7835 raft_consensus.cc:1081] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Signalling peer 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:19.266793 7606 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:33422
I20260807 19:48:19.267393 7606 raft_consensus.cc:493] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:19.267704 7606 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:19.274343 7606 raft_consensus.cc:515] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:19.277264 7606 leader_election.cc:290] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 2 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:19.277305 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:19.278095 7350 raft_consensus.cc:3055] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:19.277983 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:19.278512 7350 raft_consensus.cc:740] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:19.278864 7477 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:19.279415 7350 consensus_queue.cc:260] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:19.280809 7350 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:19.286137 7477 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 2.
I20260807 19:48:19.287977 7350 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 2.
I20260807 19:48:19.289224 7546 leader_election.cc:304] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:19.290171 7828 raft_consensus.cc:2804] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:19.291950 7828 raft_consensus.cc:697] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:19.292920 7828 consensus_queue.cc:237] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:19.301918 7174 catalog_manager.cc:5697] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 1 to 2, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 2 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:19.493037 7835 raft_consensus.cc:993] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:19.493484 7837 raft_consensus.cc:1081] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Signalling peer 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:19.495213 7606 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:33422
I20260807 19:48:19.495743 7606 raft_consensus.cc:493] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:19.496058 7606 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:19.501354 7606 raft_consensus.cc:515] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:19.503396 7606 leader_election.cc:290] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 2 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:19.504499 7350 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:19.504966 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:19.505177 7350 raft_consensus.cc:3055] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:19.505532 7350 raft_consensus.cc:740] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:19.505559 7477 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:19.506464 7350 consensus_queue.cc:260] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:19.507617 7350 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:19.512277 7477 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 2.
I20260807 19:48:19.513831 7547 leader_election.cc:304] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:19.514240 7350 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 2.
I20260807 19:48:19.514801 7828 raft_consensus.cc:2804] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:19.515300 7828 raft_consensus.cc:697] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:19.516146 7828 consensus_queue.cc:237] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:19.523315 7174 catalog_manager.cc:5697] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 1 to 2, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 2 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:19.674314 7477 raft_consensus.cc:1275] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:19.675740 7828 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:19.686246 7350 raft_consensus.cc:1275] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:19.688216 7855 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:48:20.058992 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:20.065152 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:20.065711 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:20.070684 7350 raft_consensus.cc:1275] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:20.072089 7842 consensus_queue.cc:1048] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:20.075868 7477 raft_consensus.cc:1275] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:20.078549 7842 consensus_queue.cc:1048] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:21.066852 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:21.072461 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:21.072930 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:22.074877 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:22.081575 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:22.082039 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:23.083011 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:23.089408 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:23.090067 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:24.091181 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:24.096920 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:24.097424 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:25.098484 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:25.103881 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:25.104401 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:26.105456 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:26.111658 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:26.112234 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:27.113469 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:27.118877 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:27.119344 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:28.120427 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:28.126258 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:28.126801 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:29.161986 6315 auto_leader_rebalancer-test.cc:492] The leader distribution is
I20260807 19:48:29.172899 7610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
mode: GRACEFUL
new_leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:40974
I20260807 19:48:29.174089 7610 raft_consensus.cc:606] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Received request to transfer leadership to TS 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:29.182510 7610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
mode: GRACEFUL
new_leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:40974
I20260807 19:48:29.183053 7610 raft_consensus.cc:606] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Received request to transfer leadership to TS 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:29.198561 7474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
mode: GRACEFUL
new_leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:33864
I20260807 19:48:29.199086 7474 raft_consensus.cc:606] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Received request to transfer leadership to TS 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:29.204411 7474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
mode: GRACEFUL
new_leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:33864
I20260807 19:48:29.204911 7474 raft_consensus.cc:606] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Received request to transfer leadership to TS 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:29.209794 7474 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
mode: GRACEFUL
new_leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:33864
I20260807 19:48:29.210425 7474 raft_consensus.cc:606] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Received request to transfer leadership to TS 0e2d45c99c904f8e8f5f2876e8d34d7e
I20260807 19:48:29.259935 7867 raft_consensus.cc:993] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: : Instructing follower 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.260550 7880 raft_consensus.cc:1081] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Signalling peer 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.262298 7350 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:36402
I20260807 19:48:29.262827 7350 raft_consensus.cc:493] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:29.263155 7350 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.265590 7881 raft_consensus.cc:993] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: : Instructing follower 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.266191 7870 raft_consensus.cc:1081] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Signalling peer 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.268218 7349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:36398
I20260807 19:48:29.268858 7349 raft_consensus.cc:493] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:29.269240 7349 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.270231 7350 raft_consensus.cc:515] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.272154 7350 leader_election.cc:290] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:29.273254 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:29.273527 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d113af7758c47c6840606de3d2b6cd1" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:29.274108 7474 raft_consensus.cc:3055] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:29.274164 7610 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.274588 7474 raft_consensus.cc:740] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:29.275493 7474 consensus_queue.cc:260] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.276021 7349 raft_consensus.cc:515] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.276965 7474 raft_consensus.cc:3060] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.277920 7349 leader_election.cc:290] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:29.278843 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:29.279347 7477 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.281847 7610 raft_consensus.cc:2468] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.282835 7606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab93a925d45749398c96b3f8faa369a5" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:29.283416 7606 raft_consensus.cc:3055] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:29.283548 7285 leader_election.cc:304] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:29.283838 7606 raft_consensus.cc:740] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:29.285002 7837 raft_consensus.cc:2804] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:29.284860 7606 consensus_queue.cc:260] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.285230 7474 raft_consensus.cc:2468] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.285494 7477 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.286223 7606 raft_consensus.cc:3060] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.286849 7837 raft_consensus.cc:697] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:29.287860 7287 leader_election.cc:304] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:29.287860 7837 consensus_queue.cc:237] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.288946 7883 raft_consensus.cc:2804] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:29.292896 7883 raft_consensus.cc:697] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:29.294126 7606 raft_consensus.cc:2468] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.294052 7883 consensus_queue.cc:237] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.299008 7174 catalog_manager.cc:5697] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 2 to 3, leader changed from 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194) to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 3 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:29.317461 7174 catalog_manager.cc:5697] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 2 to 3, leader changed from 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195) to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 3 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:29.632356 7881 raft_consensus.cc:993] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: : Instructing follower 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.632870 7870 raft_consensus.cc:1081] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Signalling peer 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.634806 7349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:36398
I20260807 19:48:29.635416 7349 raft_consensus.cc:493] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:29.635798 7349 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.641304 7349 raft_consensus.cc:515] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.643754 7349 leader_election.cc:290] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:29.644052 7606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:29.643985 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:29.644726 7606 raft_consensus.cc:3055] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:29.645196 7474 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.645223 7606 raft_consensus.cc:740] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:29.646229 7606 consensus_queue.cc:260] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.647094 7606 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.650043 7474 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.651073 7287 leader_election.cc:304] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:29.651695 7606 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.651906 7883 raft_consensus.cc:2804] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:29.652441 7883 raft_consensus.cc:697] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:29.653225 7883 consensus_queue.cc:237] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.660584 7174 catalog_manager.cc:5697] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 2 to 3, leader changed from 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195) to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 3 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:29.674579 7867 raft_consensus.cc:993] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: : Instructing follower 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.675038 7880 raft_consensus.cc:1081] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Signalling peer 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.676470 7349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:36402
I20260807 19:48:29.676945 7349 raft_consensus.cc:493] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:29.677222 7349 raft_consensus.cc:3060] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:29.683386 7349 raft_consensus.cc:515] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.685148 7349 leader_election.cc:290] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 2 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:29.686173 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:29.686192 7606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ef375c9422c4abaa6ad923aa48a0383" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:29.686909 7606 raft_consensus.cc:3060] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:29.688594 7474 raft_consensus.cc:3055] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:29.688936 7474 raft_consensus.cc:740] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:29.689793 7474 consensus_queue.cc:260] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.691190 7474 raft_consensus.cc:3060] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:29.693387 7606 raft_consensus.cc:2468] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 2.
I20260807 19:48:29.694677 7285 leader_election.cc:304] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:29.695369 7883 raft_consensus.cc:2804] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:29.695770 7883 raft_consensus.cc:697] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:29.696914 7474 raft_consensus.cc:2468] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 2.
I20260807 19:48:29.696969 7883 consensus_queue.cc:237] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.704895 7606 raft_consensus.cc:1275] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:29.704989 7174 catalog_manager.cc:5697] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 1 to 2, leader changed from 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194) to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 2 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:29.706485 7883 consensus_queue.cc:1048] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:29.706768 7867 raft_consensus.cc:993] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: : Instructing follower 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.707293 7880 raft_consensus.cc:1081] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Signalling peer 0e2d45c99c904f8e8f5f2876e8d34d7e to start an election
I20260807 19:48:29.708927 7349 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
from {username='slave'} at 127.0.0.1:36402
I20260807 19:48:29.709482 7349 raft_consensus.cc:493] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:29.709939 7349 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.718953 7349 raft_consensus.cc:515] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.722258 7474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:29.722899 7349 leader_election.cc:290] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Requested vote from peers 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:29.722932 7474 raft_consensus.cc:3055] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:48:29.723608 7474 raft_consensus.cc:740] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:29.724049 7606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:29.724666 7606 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.725061 7474 consensus_queue.cc:260] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.726462 7474 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:48:29.726742 7610 raft_consensus.cc:1275] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:29.728332 7883 consensus_queue.cc:1048] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:29.730836 7606 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.732084 7285 leader_election.cc:304] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e2d45c99c904f8e8f5f2876e8d34d7e, 1aa4e6fd8a854e6ab3f2b073de50781e; no voters:
I20260807 19:48:29.733130 7883 raft_consensus.cc:2804] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:48:29.733716 7883 raft_consensus.cc:697] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Leader. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:29.734884 7474 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e2d45c99c904f8e8f5f2876e8d34d7e in term 3.
I20260807 19:48:29.734753 7883 consensus_queue.cc:237] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:29.747507 7177 catalog_manager.cc:5697] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e reported cstate change: term changed from 2 to 3, leader changed from 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194) to 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193). New cstate: current_term: 3 leader_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:29.749439 7474 raft_consensus.cc:1275] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:29.751211 7884 consensus_queue.cc:1048] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:48:29.761641 7474 raft_consensus.cc:1275] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:29.763362 7909 consensus_queue.cc:1048] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:48:30.066942 7610 raft_consensus.cc:1275] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:30.068445 7837 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:30.078506 7474 raft_consensus.cc:1275] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:30.080489 7837 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:30.118158 7606 raft_consensus.cc:1275] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:30.119294 7837 consensus_queue.cc:1048] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:30.130455 7474 raft_consensus.cc:1275] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:30.132335 7909 consensus_queue.cc:1048] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:30.167176 7474 raft_consensus.cc:1275] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:30.168598 7909 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:30.178619 7606 raft_consensus.cc:1275] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Refusing update from remote peer 0e2d45c99c904f8e8f5f2876e8d34d7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:48:30.180374 7837 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:32.256556 6315 auto_leader_rebalancer-test.cc:507] The leader distribution is
I20260807 19:48:32.257407 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:32.266103 7349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:48692
I20260807 19:48:32.267253 7349 raft_consensus.cc:606] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Received request to transfer leadership to TS 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:32.270536 7349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:48692
I20260807 19:48:32.271057 7349 raft_consensus.cc:606] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Received request to transfer leadership to TS 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:32.273661 7349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:48692
I20260807 19:48:32.274149 7349 raft_consensus.cc:606] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Received request to transfer leadership to TS 7896cb60487144c19ba23a6e1eb91c84
I20260807 19:48:32.276494 7349 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9"
dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
mode: GRACEFUL
new_leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:48692
I20260807 19:48:32.276955 7349 raft_consensus.cc:606] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Received request to transfer leadership to TS 1aa4e6fd8a854e6ab3f2b073de50781e
I20260807 19:48:32.277983 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 19:48:32.278415 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:32.426918 7837 raft_consensus.cc:993] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:32.427634 7927 raft_consensus.cc:1081] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Signalling peer 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:32.429294 7610 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:33422
I20260807 19:48:32.429764 7610 raft_consensus.cc:493] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:32.430059 7610 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:32.435979 7610 raft_consensus.cc:515] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.438159 7610 leader_election.cc:290] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 4 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:32.439479 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:32.439368 7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62952c3d42b14218afacfcc9f43d71b9" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:32.440367 7477 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:32.440408 7349 raft_consensus.cc:3055] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:48:32.440950 7349 raft_consensus.cc:740] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:32.441932 7349 consensus_queue.cc:260] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.443475 7349 raft_consensus.cc:3060] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:32.448117 7477 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 4.
I20260807 19:48:32.449733 7547 leader_election.cc:304] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:32.450601 7870 raft_consensus.cc:2804] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:48:32.450747 7349 raft_consensus.cc:2468] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 4.
I20260807 19:48:32.451800 7870 raft_consensus.cc:697] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:32.452857 7870 consensus_queue.cc:237] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.460937 7177 catalog_manager.cc:5697] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 3 to 4, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 4 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:32.555680 7927 raft_consensus.cc:993] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:32.556141 7837 raft_consensus.cc:1081] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Signalling peer 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:32.557577 7477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:43792
I20260807 19:48:32.558015 7477 raft_consensus.cc:493] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:32.558274 7477 raft_consensus.cc:3060] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:32.562583 7477 raft_consensus.cc:515] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.564543 7477 leader_election.cc:290] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765)
I20260807 19:48:32.565485 7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:32.565598 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeaffc15a61427f933c6f1adad12171" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:32.566183 7610 raft_consensus.cc:3060] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:32.566176 7349 raft_consensus.cc:3055] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:32.566653 7349 raft_consensus.cc:740] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:32.567536 7349 consensus_queue.cc:260] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.568889 7349 raft_consensus.cc:3060] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:32.573370 7610 raft_consensus.cc:2468] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:32.574872 7422 leader_election.cc:304] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:32.575707 7867 raft_consensus.cc:2804] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:32.575939 7349 raft_consensus.cc:2468] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:32.577162 7867 raft_consensus.cc:697] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Becoming Leader. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:32.578148 7867 consensus_queue.cc:237] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.585033 7177 catalog_manager.cc:5697] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 reported cstate change: term changed from 1 to 2, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "7896cb60487144c19ba23a6e1eb91c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:32.674767 7837 raft_consensus.cc:993] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:32.675213 7927 raft_consensus.cc:1081] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Signalling peer 7896cb60487144c19ba23a6e1eb91c84 to start an election
I20260807 19:48:32.676831 7477 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87"
dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
from {username='slave'} at 127.0.0.1:43792
I20260807 19:48:32.677397 7477 raft_consensus.cc:493] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:32.677678 7477 raft_consensus.cc:3060] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:32.682065 7477 raft_consensus.cc:515] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:32.683589 7477 leader_election.cc:290] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Requested vote from peers 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195:45765), 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987)
I20260807 19:48:32.684376 7610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
I20260807 19:48:32.684782 7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28be4d1e918244e79b1651b304cd1d87" candidate_uuid: "7896cb60487144c19ba23a6e1eb91c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:32.684940 7610 raft_consensus.cc:3060] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:32.685293 7349 raft_consensus.cc:3055] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:32.685544 7349 raft_consensus.cc:740] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:32.686235 7349 consensus_queue.cc:260] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:32.687179 7349 raft_consensus.cc:3060] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:32.689801 7610 raft_consensus.cc:2468] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:32.690843 7422 leader_election.cc:304] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:32.691601 7867 raft_consensus.cc:2804] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:32.691996 7867 raft_consensus.cc:697] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Becoming Leader. State: Replica: 7896cb60487144c19ba23a6e1eb91c84, State: Running, Role: LEADER
I20260807 19:48:32.692564 7349 raft_consensus.cc:2468] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7896cb60487144c19ba23a6e1eb91c84 in term 2.
I20260807 19:48:32.692994 7867 consensus_queue.cc:237] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } }
I20260807 19:48:32.700310 7177 catalog_manager.cc:5697] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 reported cstate change: term changed from 1 to 2, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "7896cb60487144c19ba23a6e1eb91c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:32.864610 7837 raft_consensus.cc:993] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: : Instructing follower 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:32.865056 7837 raft_consensus.cc:1081] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Signalling peer 1aa4e6fd8a854e6ab3f2b073de50781e to start an election
I20260807 19:48:32.866504 7610 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148"
dest_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e"
from {username='slave'} at 127.0.0.1:33422
I20260807 19:48:32.866951 7610 raft_consensus.cc:493] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:32.867215 7610 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:32.871932 7610 raft_consensus.cc:515] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.873488 7610 leader_election.cc:290] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 4 election: Requested vote from peers 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987), 7896cb60487144c19ba23a6e1eb91c84 (127.6.42.194:32879)
I20260807 19:48:32.874439 7477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7896cb60487144c19ba23a6e1eb91c84"
I20260807 19:48:32.874487 7349 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28119b0c4c1d4cd0a114c7866b054148" candidate_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e"
I20260807 19:48:32.874991 7477 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:32.875066 7349 raft_consensus.cc:3055] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:48:32.875418 7349 raft_consensus.cc:740] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0e2d45c99c904f8e8f5f2876e8d34d7e, State: Running, Role: LEADER
I20260807 19:48:32.876086 7349 consensus_queue.cc:260] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.877084 7349 raft_consensus.cc:3060] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:48:32.880564 7477 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 4.
I20260807 19:48:32.881572 7547 leader_election.cc:304] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1aa4e6fd8a854e6ab3f2b073de50781e, 7896cb60487144c19ba23a6e1eb91c84; no voters:
I20260807 19:48:32.882066 7349 raft_consensus.cc:2468] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1aa4e6fd8a854e6ab3f2b073de50781e in term 4.
I20260807 19:48:32.882432 7870 raft_consensus.cc:2804] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:48:32.882867 7870 raft_consensus.cc:697] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 LEADER]: Becoming Leader. State: Replica: 1aa4e6fd8a854e6ab3f2b073de50781e, State: Running, Role: LEADER
I20260807 19:48:32.883711 7870 consensus_queue.cc:237] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } }
I20260807 19:48:32.891132 7177 catalog_manager.cc:5697] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e reported cstate change: term changed from 3 to 4, leader changed from 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193) to 1aa4e6fd8a854e6ab3f2b073de50781e (127.6.42.195). New cstate: current_term: 4 leader_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:32.957154 7477 raft_consensus.cc:1275] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:32.958214 7349 raft_consensus.cc:1275] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:32.958467 7870 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:32.959731 7929 consensus_queue.cc:1048] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:48:32.996448 7349 raft_consensus.cc:1275] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:32.997705 7867 consensus_queue.cc:1048] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:33.007253 7610 raft_consensus.cc:1275] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:33.009457 7941 consensus_queue.cc:1048] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:48:33.088181 7610 raft_consensus.cc:1275] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:33.089457 7959 consensus_queue.cc:1048] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1aa4e6fd8a854e6ab3f2b073de50781e" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 45765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:33.099465 7349 raft_consensus.cc:1275] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Refusing update from remote peer 7896cb60487144c19ba23a6e1eb91c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:48:33.102150 7960 consensus_queue.cc:1048] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:48:33.279490 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:33.284866 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:33.285367 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:33.430928 7349 raft_consensus.cc:1275] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:33.432372 7870 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2d45c99c904f8e8f5f2876e8d34d7e" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:33.443538 7477 raft_consensus.cc:1275] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Refusing update from remote peer 1aa4e6fd8a854e6ab3f2b073de50781e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260807 19:48:33.445189 7870 consensus_queue.cc:1048] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7896cb60487144c19ba23a6e1eb91c84" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:34.286394 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:34.292371 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:34.292841 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:35.293767 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:35.299363 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:35.299978 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:36.301177 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:36.306934 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:36.307396 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:37.308454 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:37.314551 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:37.315028 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:38.316018 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:38.321475 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:38.321967 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:39.322979 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:39.328446 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:39.328883 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:40.329888 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:40.335991 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:40.336503 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:41.337464 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:48:41.342952 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:41.343439 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:42.382149 6315 auto_leader_rebalancer-test.cc:518] The leader distribution is
I20260807 19:48:42.382784 6315 tablet_server.cc:179] TabletServer@127.6.42.193:0 shutting down...
I20260807 19:48:42.413045 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:42.414006 6315 tablet_replica.cc:333] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.414997 6315 raft_consensus.cc:2243] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Raft consensus shutting down.
W20260807 19:48:42.416509 7546 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 11 similar messages]
I20260807 19:48:42.417225 6315 raft_consensus.cc:2272] T d9c1f998933e43bdad3ec94a95cbe2d0 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.420418 6315 tablet_replica.cc:333] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
W20260807 19:48:42.420261 7546 consensus_peers.cc:597] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e -> Peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Couldn't send request to peer 0e2d45c99c904f8e8f5f2876e8d34d7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:42.421151 6315 raft_consensus.cc:2243] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.421798 6315 raft_consensus.cc:2272] T 62952c3d42b14218afacfcc9f43d71b9 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.423766 6315 tablet_replica.cc:333] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.424261 6315 raft_consensus.cc:2243] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.424791 6315 raft_consensus.cc:2272] T 36f3d9f167ec4e74b3b19c51f07c8292 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.426930 6315 tablet_replica.cc:333] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.427461 6315 raft_consensus.cc:2243] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.427918 6315 raft_consensus.cc:2272] T 28119b0c4c1d4cd0a114c7866b054148 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.429754 6315 tablet_replica.cc:333] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.430282 6315 raft_consensus.cc:2243] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.430825 6315 raft_consensus.cc:2272] T 28be4d1e918244e79b1651b304cd1d87 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.433431 6315 tablet_replica.cc:333] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.434221 6315 raft_consensus.cc:2243] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.434791 6315 raft_consensus.cc:2272] T c6670d21774c4fb9bd04ec36b3de9cf3 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 19:48:42.435777 7546 consensus_peers.cc:597] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e -> Peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Couldn't send request to peer 0e2d45c99c904f8e8f5f2876e8d34d7e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:42.436931 6315 tablet_replica.cc:333] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.437500 6315 raft_consensus.cc:2243] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.438344 6315 raft_consensus.cc:2272] T ab93a925d45749398c96b3f8faa369a5 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.440405 6315 tablet_replica.cc:333] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.440899 6315 raft_consensus.cc:2243] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.441360 6315 raft_consensus.cc:2272] T 0eeaffc15a61427f933c6f1adad12171 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.443395 6315 tablet_replica.cc:333] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.444072 6315 raft_consensus.cc:2243] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.445216 6315 raft_consensus.cc:2272] T 1d113af7758c47c6840606de3d2b6cd1 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.447189 6315 tablet_replica.cc:333] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e: stopping tablet replica
I20260807 19:48:42.447650 6315 raft_consensus.cc:2243] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.448524 6315 raft_consensus.cc:2272] T 9ef375c9422c4abaa6ad923aa48a0383 P 0e2d45c99c904f8e8f5f2876e8d34d7e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.481473 6315 tablet_server.cc:196] TabletServer@127.6.42.193:0 shutdown complete.
W20260807 19:48:42.482259 7422 consensus_peers.cc:597] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 -> Peer 0e2d45c99c904f8e8f5f2876e8d34d7e (127.6.42.193:33987): Couldn't send request to peer 0e2d45c99c904f8e8f5f2876e8d34d7e. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:33987: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:42.503911 6315 tablet_server.cc:179] TabletServer@127.6.42.194:0 shutting down...
I20260807 19:48:42.533836 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:42.534550 6315 tablet_replica.cc:333] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.535156 6315 raft_consensus.cc:2243] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.535737 6315 raft_consensus.cc:2272] T 62952c3d42b14218afacfcc9f43d71b9 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.537773 6315 tablet_replica.cc:333] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.538318 6315 raft_consensus.cc:2243] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.538782 6315 raft_consensus.cc:2272] T ab93a925d45749398c96b3f8faa369a5 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.540668 6315 tablet_replica.cc:333] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.541189 6315 raft_consensus.cc:2243] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.541992 6315 raft_consensus.cc:2272] T 0eeaffc15a61427f933c6f1adad12171 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.543725 6315 tablet_replica.cc:333] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.544216 6315 raft_consensus.cc:2243] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.544730 6315 raft_consensus.cc:2272] T d9c1f998933e43bdad3ec94a95cbe2d0 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.546579 6315 tablet_replica.cc:333] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.547047 6315 raft_consensus.cc:2243] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.547519 6315 raft_consensus.cc:2272] T 1d113af7758c47c6840606de3d2b6cd1 P 7896cb60487144c19ba23a6e1eb91c84 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.549207 6315 tablet_replica.cc:333] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.549691 6315 raft_consensus.cc:2243] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.550082 6315 raft_consensus.cc:2272] T 9ef375c9422c4abaa6ad923aa48a0383 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.551760 6315 tablet_replica.cc:333] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.552228 6315 raft_consensus.cc:2243] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.553062 6315 raft_consensus.cc:2272] T 28be4d1e918244e79b1651b304cd1d87 P 7896cb60487144c19ba23a6e1eb91c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.554884 6315 tablet_replica.cc:333] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.555346 6315 raft_consensus.cc:2243] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.555743 6315 raft_consensus.cc:2272] T c6670d21774c4fb9bd04ec36b3de9cf3 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.557458 6315 tablet_replica.cc:333] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.557912 6315 raft_consensus.cc:2243] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.558362 6315 raft_consensus.cc:2272] T 28119b0c4c1d4cd0a114c7866b054148 P 7896cb60487144c19ba23a6e1eb91c84 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.560171 6315 tablet_replica.cc:333] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84: stopping tablet replica
I20260807 19:48:42.560614 6315 raft_consensus.cc:2243] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.561381 6315 raft_consensus.cc:2272] T 36f3d9f167ec4e74b3b19c51f07c8292 P 7896cb60487144c19ba23a6e1eb91c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.591545 6315 tablet_server.cc:196] TabletServer@127.6.42.194:0 shutdown complete.
I20260807 19:48:42.612627 6315 tablet_server.cc:179] TabletServer@127.6.42.195:0 shutting down...
I20260807 19:48:42.638453 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:42.639400 6315 tablet_replica.cc:333] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.640084 6315 raft_consensus.cc:2243] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.640583 6315 raft_consensus.cc:2272] T 36f3d9f167ec4e74b3b19c51f07c8292 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.642494 6315 tablet_replica.cc:333] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.643042 6315 raft_consensus.cc:2243] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.643852 6315 raft_consensus.cc:2272] T c6670d21774c4fb9bd04ec36b3de9cf3 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.645867 6315 tablet_replica.cc:333] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.646373 6315 raft_consensus.cc:2243] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.647176 6315 raft_consensus.cc:2272] T 28119b0c4c1d4cd0a114c7866b054148 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.649164 6315 tablet_replica.cc:333] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.649708 6315 raft_consensus.cc:2243] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.650157 6315 raft_consensus.cc:2272] T 28be4d1e918244e79b1651b304cd1d87 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.651927 6315 tablet_replica.cc:333] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.652463 6315 raft_consensus.cc:2243] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.652956 6315 raft_consensus.cc:2272] T d9c1f998933e43bdad3ec94a95cbe2d0 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.654709 6315 tablet_replica.cc:333] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.655207 6315 raft_consensus.cc:2243] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.656020 6315 raft_consensus.cc:2272] T 62952c3d42b14218afacfcc9f43d71b9 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.657976 6315 tablet_replica.cc:333] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.658486 6315 raft_consensus.cc:2243] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.659003 6315 raft_consensus.cc:2272] T 1d113af7758c47c6840606de3d2b6cd1 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.660842 6315 tablet_replica.cc:333] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.661350 6315 raft_consensus.cc:2243] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.661767 6315 raft_consensus.cc:2272] T 9ef375c9422c4abaa6ad923aa48a0383 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.663565 6315 tablet_replica.cc:333] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.664096 6315 raft_consensus.cc:2243] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.664606 6315 raft_consensus.cc:2272] T 0eeaffc15a61427f933c6f1adad12171 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.666518 6315 tablet_replica.cc:333] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e: stopping tablet replica
I20260807 19:48:42.667013 6315 raft_consensus.cc:2243] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:42.667467 6315 raft_consensus.cc:2272] T ab93a925d45749398c96b3f8faa369a5 P 1aa4e6fd8a854e6ab3f2b073de50781e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.697532 6315 tablet_server.cc:196] TabletServer@127.6.42.195:0 shutdown complete.
I20260807 19:48:42.721740 6315 master.cc:562] Master@127.6.42.254:40889 shutting down...
I20260807 19:48:42.748484 6315 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:42.749063 6315 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:42.749357 6315 tablet_replica.cc:333] T 00000000000000000000000000000000 P d40f6e78a0294925a132c2514b007686: stopping tablet replica
I20260807 19:48:42.768875 6315 master.cc:584] Master@127.6.42.254:40889 shutdown complete.
0e2d45c99c904f8e8f5f2876e8d34d7e 5
1aa4e6fd8a854e6ab3f2b073de50781e 1
7896cb60487144c19ba23a6e1eb91c84 4
0e2d45c99c904f8e8f5f2876e8d34d7e 3
1aa4e6fd8a854e6ab3f2b073de50781e 3
7896cb60487144c19ba23a6e1eb91c84 4
0e2d45c99c904f8e8f5f2876e8d34d7e 8
1aa4e6fd8a854e6ab3f2b073de50781e 1
7896cb60487144c19ba23a6e1eb91c84 1
0e2d45c99c904f8e8f5f2876e8d34d7e 4
1aa4e6fd8a854e6ab3f2b073de50781e 3
7896cb60487144c19ba23a6e1eb91c84 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28746 ms)
[ RUN ] LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass
I20260807 19:48:42.811739 6315 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.42.254:35721
I20260807 19:48:42.813030 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:42.819267 7992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:42.819288 7993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:42.819651 7998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:42.820629 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:42.821631 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:42.821821 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:42.821979 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132122821961 us; error 0 us; skew 500 ppm
I20260807 19:48:42.822522 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:42.825520 6315 webserver.cc:533] Webserver started at http://127.6.42.254:38705/ using document root <none> and password file <none>
I20260807 19:48:42.826046 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:42.826246 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:42.826529 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:42.827839 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/master-0-root/instance:
uuid: "20fc144bd2764ac283bb04a7da62d2ce"
format_stamp: "Formatted at 2026-08-07 19:48:42 on dist-test-slave-lkx2"
I20260807 19:48:42.833094 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:48:42.837127 8010 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:42.838275 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:48:42.838619 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/master-0-root
uuid: "20fc144bd2764ac283bb04a7da62d2ce"
format_stamp: "Formatted at 2026-08-07 19:48:42 on dist-test-slave-lkx2"
I20260807 19:48:42.838922 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:42.863152 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:42.864738 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:42.907806 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.254:35721
I20260807 19:48:42.907894 8084 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.254:35721 every 8 connection(s)
I20260807 19:48:42.911870 8086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:42.922580 8086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce: Bootstrap starting.
I20260807 19:48:42.927632 8086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:42.932552 8086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce: No bootstrap required, opened a new log
I20260807 19:48:42.935240 8086 raft_consensus.cc:359] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc144bd2764ac283bb04a7da62d2ce" member_type: VOTER }
I20260807 19:48:42.935750 8086 raft_consensus.cc:385] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:42.936053 8086 raft_consensus.cc:740] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20fc144bd2764ac283bb04a7da62d2ce, State: Initialized, Role: FOLLOWER
I20260807 19:48:42.936798 8086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc144bd2764ac283bb04a7da62d2ce" member_type: VOTER }
I20260807 19:48:42.937395 8086 raft_consensus.cc:399] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:48:42.937738 8086 raft_consensus.cc:493] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:48:42.938045 8086 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:42.943166 8086 raft_consensus.cc:515] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc144bd2764ac283bb04a7da62d2ce" member_type: VOTER }
I20260807 19:48:42.943758 8086 leader_election.cc:304] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20fc144bd2764ac283bb04a7da62d2ce; no voters:
I20260807 19:48:42.945096 8086 leader_election.cc:290] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:48:42.945467 8091 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:42.946928 8091 raft_consensus.cc:697] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 1 LEADER]: Becoming Leader. State: Replica: 20fc144bd2764ac283bb04a7da62d2ce, State: Running, Role: LEADER
I20260807 19:48:42.947824 8091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc144bd2764ac283bb04a7da62d2ce" member_type: VOTER }
I20260807 19:48:42.948166 8086 sys_catalog.cc:565] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:48:42.953168 8096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20fc144bd2764ac283bb04a7da62d2ce. Latest consensus state: current_term: 1 leader_uuid: "20fc144bd2764ac283bb04a7da62d2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc144bd2764ac283bb04a7da62d2ce" member_type: VOTER } }
I20260807 19:48:42.953473 8093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20fc144bd2764ac283bb04a7da62d2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc144bd2764ac283bb04a7da62d2ce" member_type: VOTER } }
I20260807 19:48:42.954015 8096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:42.954186 8093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:42.963306 8106 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 19:48:42.968576 8106 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 19:48:42.969875 6315 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 19:48:42.978758 8106 catalog_manager.cc:1383] Generated new cluster ID: 76a4eb6c81154f2aa42b00a193210c24
I20260807 19:48:42.979126 8106 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 19:48:43.003100 8106 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 19:48:43.004623 8106 catalog_manager.cc:1540] Loading token signing keys...
I20260807 19:48:43.023068 8106 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce: Generated new TSK 0
I20260807 19:48:43.024001 8106 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 19:48:43.037889 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:43.045464 8121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:43.046396 8128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:43.048149 8130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:43.049888 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:43.051010 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:43.051215 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:43.051378 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132123051359 us; error 0 us; skew 500 ppm
I20260807 19:48:43.051949 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:43.054780 6315 webserver.cc:533] Webserver started at http://127.6.42.193:45697/ using document root <none> and password file <none>
I20260807 19:48:43.055320 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:43.055522 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:43.055781 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:43.057030 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-0-root/instance:
uuid: "e287c74a50ec40ecadff065dd8be7c5a"
format_stamp: "Formatted at 2026-08-07 19:48:43 on dist-test-slave-lkx2"
I20260807 19:48:43.062316 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:48:43.065932 8138 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.066687 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 19:48:43.067397 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-0-root
uuid: "e287c74a50ec40ecadff065dd8be7c5a"
format_stamp: "Formatted at 2026-08-07 19:48:43 on dist-test-slave-lkx2"
I20260807 19:48:43.067689 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:43.086450 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:43.087808 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:43.089561 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:43.092406 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:43.092662 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.092911 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:43.093065 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.141997 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.193:42663
I20260807 19:48:43.142205 8232 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.193:42663 every 8 connection(s)
I20260807 19:48:43.147858 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:43.155375 8245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:43.157030 8246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:43.158952 8248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:43.160359 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:43.161485 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:43.161756 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:43.161864 8235 heartbeater.cc:344] Connected to a master server at 127.6.42.254:35721
I20260807 19:48:43.161995 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132123161971 us; error 0 us; skew 500 ppm
I20260807 19:48:43.162315 8235 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:43.162807 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:43.163278 8235 heartbeater.cc:507] Master 127.6.42.254:35721 requested a full tablet report, sending...
I20260807 19:48:43.165691 6315 webserver.cc:533] Webserver started at http://127.6.42.194:34627/ using document root <none> and password file <none>
I20260807 19:48:43.165978 8032 ts_manager.cc:194] Registered new tserver with Master: e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663)
I20260807 19:48:43.166249 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:43.166507 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:43.166769 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:43.167892 8032 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56122
I20260807 19:48:43.168087 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-1-root/instance:
uuid: "3c2790d840c74645a489a5918a9ad5be"
format_stamp: "Formatted at 2026-08-07 19:48:43 on dist-test-slave-lkx2"
I20260807 19:48:43.173794 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:48:43.177345 8257 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.178218 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 19:48:43.178507 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-1-root
uuid: "3c2790d840c74645a489a5918a9ad5be"
format_stamp: "Formatted at 2026-08-07 19:48:43 on dist-test-slave-lkx2"
I20260807 19:48:43.178783 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:43.197548 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:43.198813 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:43.200487 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:43.202879 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:43.203083 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.203312 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:43.203467 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.249665 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.194:34513
I20260807 19:48:43.249765 8358 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.194:34513 every 8 connection(s)
I20260807 19:48:43.254614 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:43.262471 8364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:43.263557 8366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:43.265672 8370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:43.269465 8359 heartbeater.cc:344] Connected to a master server at 127.6.42.254:35721
I20260807 19:48:43.269943 8359 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:43.270347 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:43.270871 8359 heartbeater.cc:507] Master 127.6.42.254:35721 requested a full tablet report, sending...
I20260807 19:48:43.271466 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:43.271757 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:43.271962 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132123271940 us; error 0 us; skew 500 ppm
I20260807 19:48:43.272697 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:43.273360 8032 ts_manager.cc:194] Registered new tserver with Master: 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:43.275076 8032 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56126
I20260807 19:48:43.275734 6315 webserver.cc:533] Webserver started at http://127.6.42.195:45543/ using document root <none> and password file <none>
I20260807 19:48:43.276413 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:43.276664 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:43.277002 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:43.278393 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-2-root/instance:
uuid: "27887cbd32644b5286ac76a1a25f550f"
format_stamp: "Formatted at 2026-08-07 19:48:43 on dist-test-slave-lkx2"
I20260807 19:48:43.283375 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:48:43.287014 8378 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.287917 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:48:43.288232 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-2-root
uuid: "27887cbd32644b5286ac76a1a25f550f"
format_stamp: "Formatted at 2026-08-07 19:48:43 on dist-test-slave-lkx2"
I20260807 19:48:43.288580 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass.1786132061667097-6315-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:43.313524 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:43.314854 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:43.316385 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:43.318652 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:43.318854 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.319079 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:43.319231 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:43.363514 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.195:37985
I20260807 19:48:43.363698 8488 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.195:37985 every 8 connection(s)
I20260807 19:48:43.378469 8490 heartbeater.cc:344] Connected to a master server at 127.6.42.254:35721
I20260807 19:48:43.378862 8490 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:43.379664 8490 heartbeater.cc:507] Master 127.6.42.254:35721 requested a full tablet report, sending...
I20260807 19:48:43.381878 8032 ts_manager.cc:194] Registered new tserver with Master: 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:43.381958 6315 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014992453s
I20260807 19:48:43.383309 8032 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56132
I20260807 19:48:43.410858 8032 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56146:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:48:43.413590 8032 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:48:43.475060 8428 tablet_service.cc:1511] Processing CreateTablet for tablet c5ea32be48a34c68878ed7f40aa64557 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION VALUES < 357913941
I20260807 19:48:43.474999 8427 tablet_service.cc:1511] Processing CreateTablet for tablet 44877db0c4c54c1d9ee4a50184c4e63a (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20260807 19:48:43.476722 8428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ea32be48a34c68878ed7f40aa64557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.477418 8427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44877db0c4c54c1d9ee4a50184c4e63a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.480365 8425 tablet_service.cc:1511] Processing CreateTablet for tablet b921e5ba58f841808417b01e7f5ac497 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20260807 19:48:43.481827 8425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b921e5ba58f841808417b01e7f5ac497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.485769 8424 tablet_service.cc:1511] Processing CreateTablet for tablet 2e0d11c546d94f48b9eb32a0d6d74652 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20260807 19:48:43.487210 8424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e0d11c546d94f48b9eb32a0d6d74652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.488271 8178 tablet_service.cc:1511] Processing CreateTablet for tablet c5ea32be48a34c68878ed7f40aa64557 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION VALUES < 357913941
I20260807 19:48:43.489689 8178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ea32be48a34c68878ed7f40aa64557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.491449 8177 tablet_service.cc:1511] Processing CreateTablet for tablet 44877db0c4c54c1d9ee4a50184c4e63a (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20260807 19:48:43.493633 8176 tablet_service.cc:1511] Processing CreateTablet for tablet b921e5ba58f841808417b01e7f5ac497 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20260807 19:48:43.495325 8177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44877db0c4c54c1d9ee4a50184c4e63a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.495568 8174 tablet_service.cc:1511] Processing CreateTablet for tablet 2e0d11c546d94f48b9eb32a0d6d74652 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20260807 19:48:43.496153 8421 tablet_service.cc:1511] Processing CreateTablet for tablet 254a1cd9838545e58769e0f60d8ef4f5 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20260807 19:48:43.497570 8176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b921e5ba58f841808417b01e7f5ac497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.497745 8421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 254a1cd9838545e58769e0f60d8ef4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.501511 8513 tablet_bootstrap.cc:492] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: Bootstrap starting.
I20260807 19:48:43.501914 8173 tablet_service.cc:1511] Processing CreateTablet for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20260807 19:48:43.503384 8173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.506994 8174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e0d11c546d94f48b9eb32a0d6d74652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.508481 8513 tablet_bootstrap.cc:654] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.491992 8423 tablet_service.cc:1511] Processing CreateTablet for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20260807 19:48:43.509793 8423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.510151 8172 tablet_service.cc:1511] Processing CreateTablet for tablet 254a1cd9838545e58769e0f60d8ef4f5 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20260807 19:48:43.511688 8172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 254a1cd9838545e58769e0f60d8ef4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.575867 8518 tablet_bootstrap.cc:492] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: Bootstrap starting.
I20260807 19:48:43.583081 8301 tablet_service.cc:1511] Processing CreateTablet for tablet c5ea32be48a34c68878ed7f40aa64557 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION VALUES < 357913941
I20260807 19:48:43.584621 8518 tablet_bootstrap.cc:654] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.584867 8292 tablet_service.cc:1511] Processing CreateTablet for tablet 254a1cd9838545e58769e0f60d8ef4f5 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20260807 19:48:43.585605 8300 tablet_service.cc:1511] Processing CreateTablet for tablet 44877db0c4c54c1d9ee4a50184c4e63a (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20260807 19:48:43.588042 8299 tablet_service.cc:1511] Processing CreateTablet for tablet b921e5ba58f841808417b01e7f5ac497 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20260807 19:48:43.590528 8298 tablet_service.cc:1511] Processing CreateTablet for tablet 2e0d11c546d94f48b9eb32a0d6d74652 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20260807 19:48:43.593307 8295 tablet_service.cc:1511] Processing CreateTablet for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 (DEFAULT_TABLE table=test-workload [id=57105b6e7499487d987ae01c97ab83a0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20260807 19:48:43.595201 8301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ea32be48a34c68878ed7f40aa64557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.597122 8300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44877db0c4c54c1d9ee4a50184c4e63a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.600134 8298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e0d11c546d94f48b9eb32a0d6d74652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.604791 8299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b921e5ba58f841808417b01e7f5ac497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.608809 8295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.624727 8292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 254a1cd9838545e58769e0f60d8ef4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:43.649874 8513 tablet_bootstrap.cc:492] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: No bootstrap required, opened a new log
I20260807 19:48:43.660740 8513 ts_tablet_manager.cc:1403] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: Time spent bootstrapping tablet: real 0.160s user 0.058s sys 0.056s
I20260807 19:48:43.675755 8513 raft_consensus.cc:359] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.676841 8513 raft_consensus.cc:385] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.677192 8513 raft_consensus.cc:740] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.677922 8518 tablet_bootstrap.cc:492] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: No bootstrap required, opened a new log
I20260807 19:48:43.677984 8513 consensus_queue.cc:260] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.679165 8520 tablet_bootstrap.cc:492] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be: Bootstrap starting.
I20260807 19:48:43.680444 8490 heartbeater.cc:499] Master 127.6.42.254:35721 was elected leader, sending a full tablet report...
I20260807 19:48:43.680963 8513 ts_tablet_manager.cc:1434] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: Time spent starting tablet: real 0.019s user 0.006s sys 0.001s
I20260807 19:48:43.681972 8513 tablet_bootstrap.cc:492] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: Bootstrap starting.
I20260807 19:48:43.684229 8518 ts_tablet_manager.cc:1403] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: Time spent bootstrapping tablet: real 0.109s user 0.015s sys 0.047s
I20260807 19:48:43.687425 8518 raft_consensus.cc:359] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.688164 8518 raft_consensus.cc:385] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.688463 8513 tablet_bootstrap.cc:654] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.692167 8520 tablet_bootstrap.cc:654] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.688501 8518 raft_consensus.cc:740] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.693336 8518 consensus_queue.cc:260] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.694720 8513 tablet_bootstrap.cc:492] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: No bootstrap required, opened a new log
I20260807 19:48:43.695216 8513 ts_tablet_manager.cc:1403] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.007s
I20260807 19:48:43.698081 8513 raft_consensus.cc:359] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.698758 8513 raft_consensus.cc:385] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.699060 8513 raft_consensus.cc:740] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.699822 8513 consensus_queue.cc:260] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.701270 8520 tablet_bootstrap.cc:492] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be: No bootstrap required, opened a new log
I20260807 19:48:43.701808 8520 ts_tablet_manager.cc:1403] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.004s
I20260807 19:48:43.702142 8513 ts_tablet_manager.cc:1434] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260807 19:48:43.703091 8513 tablet_bootstrap.cc:492] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f: Bootstrap starting.
I20260807 19:48:43.704964 8520 raft_consensus.cc:359] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.705803 8520 raft_consensus.cc:385] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.706151 8520 raft_consensus.cc:740] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2790d840c74645a489a5918a9ad5be, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.708096 8235 heartbeater.cc:499] Master 127.6.42.254:35721 was elected leader, sending a full tablet report...
I20260807 19:48:43.708639 8518 ts_tablet_manager.cc:1434] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: Time spent starting tablet: real 0.024s user 0.008s sys 0.012s
I20260807 19:48:43.709616 8513 tablet_bootstrap.cc:654] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.709602 8518 tablet_bootstrap.cc:492] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: Bootstrap starting.
I20260807 19:48:43.716607 8518 tablet_bootstrap.cc:654] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.718223 8513 tablet_bootstrap.cc:492] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f: No bootstrap required, opened a new log
I20260807 19:48:43.718711 8513 ts_tablet_manager.cc:1403] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.005s
I20260807 19:48:43.718811 8520 consensus_queue.cc:260] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.722074 8513 raft_consensus.cc:359] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.722899 8513 raft_consensus.cc:385] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.723240 8513 raft_consensus.cc:740] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.724292 8513 consensus_queue.cc:260] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.733626 8520 ts_tablet_manager.cc:1434] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be: Time spent starting tablet: real 0.032s user 0.014s sys 0.005s
I20260807 19:48:43.733861 8518 tablet_bootstrap.cc:492] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: No bootstrap required, opened a new log
I20260807 19:48:43.734412 8518 ts_tablet_manager.cc:1403] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.020s
I20260807 19:48:43.734678 8513 ts_tablet_manager.cc:1434] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20260807 19:48:43.735577 8513 tablet_bootstrap.cc:492] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f: Bootstrap starting.
I20260807 19:48:43.736147 8359 heartbeater.cc:499] Master 127.6.42.254:35721 was elected leader, sending a full tablet report...
I20260807 19:48:43.737309 8518 raft_consensus.cc:359] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.738077 8518 raft_consensus.cc:385] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.738375 8518 raft_consensus.cc:740] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.739204 8518 consensus_queue.cc:260] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.740871 8521 raft_consensus.cc:493] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:43.741381 8521 raft_consensus.cc:515] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.742887 8513 tablet_bootstrap.cc:654] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.744215 8518 ts_tablet_manager.cc:1434] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:48:43.745353 8518 tablet_bootstrap.cc:492] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: Bootstrap starting.
I20260807 19:48:43.751742 8518 tablet_bootstrap.cc:654] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.754094 8534 raft_consensus.cc:493] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:43.754635 8534 raft_consensus.cc:515] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.767023 8534 leader_election.cc:290] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663)
I20260807 19:48:43.768878 8521 leader_election.cc:290] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:43.775681 8518 tablet_bootstrap.cc:492] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: No bootstrap required, opened a new log
I20260807 19:48:43.776269 8518 ts_tablet_manager.cc:1403] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.014s
I20260807 19:48:43.777019 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a" is_pre_election: true
I20260807 19:48:43.777045 8194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a" is_pre_election: true
I20260807 19:48:43.777923 8194 raft_consensus.cc:2468] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 0.
I20260807 19:48:43.777925 8196 raft_consensus.cc:2468] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 0.
I20260807 19:48:43.779479 8518 raft_consensus.cc:359] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.780180 8518 raft_consensus.cc:385] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.780493 8520 tablet_bootstrap.cc:492] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: Bootstrap starting.
I20260807 19:48:43.780507 8518 raft_consensus.cc:740] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.782559 8518 consensus_queue.cc:260] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.783288 8513 tablet_bootstrap.cc:492] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f: No bootstrap required, opened a new log
I20260807 19:48:43.783793 8513 ts_tablet_manager.cc:1403] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f: Time spent bootstrapping tablet: real 0.048s user 0.029s sys 0.014s
I20260807 19:48:43.787352 8518 ts_tablet_manager.cc:1434] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260807 19:48:43.787345 8513 raft_consensus.cc:359] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.788306 8513 raft_consensus.cc:385] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.788513 8518 tablet_bootstrap.cc:492] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a: Bootstrap starting.
I20260807 19:48:43.788650 8513 raft_consensus.cc:740] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.788584 8382 leader_election.cc:304] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:43.789898 8382 leader_election.cc:304] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:43.789589 8513 consensus_queue.cc:260] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.790850 8534 raft_consensus.cc:2804] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:43.791224 8534 raft_consensus.cc:493] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:43.791566 8534 raft_consensus.cc:3060] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.797704 8539 raft_consensus.cc:2804] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:43.798151 8539 raft_consensus.cc:493] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:43.798239 8520 tablet_bootstrap.cc:654] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.798514 8539 raft_consensus.cc:3060] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.800387 8513 ts_tablet_manager.cc:1434] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260807 19:48:43.801411 8513 tablet_bootstrap.cc:492] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: Bootstrap starting.
I20260807 19:48:43.798570 8518 tablet_bootstrap.cc:654] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.808245 8513 tablet_bootstrap.cc:654] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.809652 8539 raft_consensus.cc:515] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.812649 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
I20260807 19:48:43.813266 8196 raft_consensus.cc:3060] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.816908 8534 raft_consensus.cc:515] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.820188 8196 raft_consensus.cc:2468] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 1.
I20260807 19:48:43.825659 8382 leader_election.cc:304] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:43.826550 8521 raft_consensus.cc:2804] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:43.823961 8518 tablet_bootstrap.cc:492] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a: No bootstrap required, opened a new log
I20260807 19:48:43.828577 8534 leader_election.cc:290] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Requested vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663)
I20260807 19:48:43.829557 8318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
I20260807 19:48:43.830549 8518 ts_tablet_manager.cc:1403] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a: Time spent bootstrapping tablet: real 0.042s user 0.010s sys 0.003s
I20260807 19:48:43.830988 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
I20260807 19:48:43.831298 8313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
W20260807 19:48:43.832253 8379 leader_election.cc:343] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:43.832875 8311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:43.833335 8196 raft_consensus.cc:3060] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.834388 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:43.834753 8518 raft_consensus.cc:359] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.835495 8518 raft_consensus.cc:385] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.835800 8518 raft_consensus.cc:740] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.839017 8518 consensus_queue.cc:260] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
W20260807 19:48:43.839844 8379 leader_election.cc:343] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:43.841193 8379 leader_election.cc:343] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:43.841943 8379 leader_election.cc:343] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:43.843104 8518 ts_tablet_manager.cc:1434] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260807 19:48:43.844166 8518 tablet_bootstrap.cc:492] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a: Bootstrap starting.
I20260807 19:48:43.845121 8539 leader_election.cc:290] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Requested vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:43.845695 8196 raft_consensus.cc:2468] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 1.
I20260807 19:48:43.847625 8382 leader_election.cc:304] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters: 3c2790d840c74645a489a5918a9ad5be
I20260807 19:48:43.849046 8539 raft_consensus.cc:2804] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:43.849953 8539 raft_consensus.cc:697] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Becoming Leader. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Running, Role: LEADER
I20260807 19:48:43.851177 8539 consensus_queue.cc:237] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.854982 8520 tablet_bootstrap.cc:492] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: No bootstrap required, opened a new log
I20260807 19:48:43.856065 8520 ts_tablet_manager.cc:1403] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: Time spent bootstrapping tablet: real 0.076s user 0.029s sys 0.016s
I20260807 19:48:43.856858 8521 raft_consensus.cc:697] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Becoming Leader. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Running, Role: LEADER
I20260807 19:48:43.855226 8518 tablet_bootstrap.cc:654] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.857893 8521 consensus_queue.cc:237] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.858314 8513 tablet_bootstrap.cc:492] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: No bootstrap required, opened a new log
I20260807 19:48:43.859087 8513 ts_tablet_manager.cc:1403] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: Time spent bootstrapping tablet: real 0.058s user 0.019s sys 0.037s
I20260807 19:48:43.859408 8520 raft_consensus.cc:359] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.860188 8520 raft_consensus.cc:385] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.860558 8520 raft_consensus.cc:740] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2790d840c74645a489a5918a9ad5be, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.861884 8520 consensus_queue.cc:260] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.864315 8513 raft_consensus.cc:359] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.865087 8513 raft_consensus.cc:385] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.865387 8513 raft_consensus.cc:740] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.866278 8513 consensus_queue.cc:260] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.868440 8513 ts_tablet_manager.cc:1434] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:48:43.869499 8513 tablet_bootstrap.cc:492] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f: Bootstrap starting.
I20260807 19:48:43.870338 8520 ts_tablet_manager.cc:1434] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260807 19:48:43.871243 8518 tablet_bootstrap.cc:492] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a: No bootstrap required, opened a new log
I20260807 19:48:43.871891 8518 ts_tablet_manager.cc:1403] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.009s
I20260807 19:48:43.871352 8520 tablet_bootstrap.cc:492] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be: Bootstrap starting.
I20260807 19:48:43.875065 8518 raft_consensus.cc:359] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.875913 8513 tablet_bootstrap.cc:654] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.876071 8518 raft_consensus.cc:385] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.876628 8518 raft_consensus.cc:740] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.877723 8518 consensus_queue.cc:260] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.882984 8518 ts_tablet_manager.cc:1434] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260807 19:48:43.884027 8520 tablet_bootstrap.cc:654] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.884121 8518 tablet_bootstrap.cc:492] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a: Bootstrap starting.
I20260807 19:48:43.887852 8527 raft_consensus.cc:493] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:43.888200 8513 tablet_bootstrap.cc:492] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f: No bootstrap required, opened a new log
I20260807 19:48:43.888837 8513 ts_tablet_manager.cc:1403] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20260807 19:48:43.888401 8527 raft_consensus.cc:515] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.891311 8527 leader_election.cc:290] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:43.893662 8518 tablet_bootstrap.cc:654] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.900720 8520 tablet_bootstrap.cc:492] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be: No bootstrap required, opened a new log
I20260807 19:48:43.901294 8520 ts_tablet_manager.cc:1403] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.012s
I20260807 19:48:43.900908 8513 raft_consensus.cc:359] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.906422 8520 raft_consensus.cc:359] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.906168 8032 catalog_manager.cc:5697] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f reported cstate change: term changed from 0 to 1, leader changed from <none> to 27887cbd32644b5286ac76a1a25f550f (127.6.42.195). New cstate: current_term: 1 leader_uuid: "27887cbd32644b5286ac76a1a25f550f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:43.907989 8513 raft_consensus.cc:385] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.908860 8513 raft_consensus.cc:740] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.909704 8513 consensus_queue.cc:260] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.912057 8513 ts_tablet_manager.cc:1434] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260807 19:48:43.908025 8520 raft_consensus.cc:385] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.912798 8520 raft_consensus.cc:740] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2790d840c74645a489a5918a9ad5be, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.913643 8520 consensus_queue.cc:260] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.916181 8520 ts_tablet_manager.cc:1434] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20260807 19:48:43.916209 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254a1cd9838545e58769e0f60d8ef4f5" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
I20260807 19:48:43.917364 8520 tablet_bootstrap.cc:492] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be: Bootstrap starting.
I20260807 19:48:43.918447 8518 tablet_bootstrap.cc:492] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a: No bootstrap required, opened a new log
I20260807 19:48:43.918951 8518 ts_tablet_manager.cc:1403] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a: Time spent bootstrapping tablet: real 0.035s user 0.011s sys 0.007s
I20260807 19:48:43.922116 8518 raft_consensus.cc:359] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.922894 8518 raft_consensus.cc:385] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.923198 8518 raft_consensus.cc:740] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.923888 8520 tablet_bootstrap.cc:654] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.923887 8518 consensus_queue.cc:260] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
W20260807 19:48:43.925529 8140 leader_election.cc:343] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:43.927186 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254a1cd9838545e58769e0f60d8ef4f5" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27887cbd32644b5286ac76a1a25f550f" is_pre_election: true
I20260807 19:48:43.927908 8450 raft_consensus.cc:2468] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 0.
I20260807 19:48:43.928225 8518 ts_tablet_manager.cc:1434] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:48:43.931572 8140 leader_election.cc:304] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters: 3c2790d840c74645a489a5918a9ad5be
I20260807 19:48:43.932637 8527 raft_consensus.cc:2804] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:43.933027 8527 raft_consensus.cc:493] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:43.933354 8527 raft_consensus.cc:3060] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.937630 8520 tablet_bootstrap.cc:492] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be: No bootstrap required, opened a new log
I20260807 19:48:43.936879 8030 catalog_manager.cc:5697] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f reported cstate change: term changed from 0 to 1, leader changed from <none> to 27887cbd32644b5286ac76a1a25f550f (127.6.42.195). New cstate: current_term: 1 leader_uuid: "27887cbd32644b5286ac76a1a25f550f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:43.938125 8520 ts_tablet_manager.cc:1403] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be: Time spent bootstrapping tablet: real 0.021s user 0.011s sys 0.008s
I20260807 19:48:43.941224 8527 raft_consensus.cc:515] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.941228 8520 raft_consensus.cc:359] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.942430 8520 raft_consensus.cc:385] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.942771 8520 raft_consensus.cc:740] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2790d840c74645a489a5918a9ad5be, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.943802 8527 leader_election.cc:290] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 election: Requested vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:43.943632 8520 consensus_queue.cc:260] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.944428 8527 raft_consensus.cc:493] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:43.944898 8527 raft_consensus.cc:515] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.946933 8527 leader_election.cc:290] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:43.951759 8520 ts_tablet_manager.cc:1434] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260807 19:48:43.953122 8520 tablet_bootstrap.cc:492] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be: Bootstrap starting.
I20260807 19:48:43.956311 8521 raft_consensus.cc:493] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:43.956966 8521 raft_consensus.cc:515] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.959326 8521 leader_election.cc:290] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:43.960239 8520 tablet_bootstrap.cc:654] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:43.961243 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a" is_pre_election: true
W20260807 19:48:43.961683 8109 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:43.962013 8196 raft_consensus.cc:2468] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 0.
I20260807 19:48:43.964423 8382 leader_election.cc:304] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:43.965353 8521 raft_consensus.cc:2804] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:43.965705 8521 raft_consensus.cc:493] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:43.965991 8521 raft_consensus.cc:3060] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.969614 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
I20260807 19:48:43.970297 8310 raft_consensus.cc:2468] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 0.
I20260807 19:48:43.974612 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254a1cd9838545e58769e0f60d8ef4f5" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:43.974457 8521 raft_consensus.cc:515] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.975425 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254a1cd9838545e58769e0f60d8ef4f5" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
I20260807 19:48:43.976004 8450 raft_consensus.cc:3060] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.976794 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
I20260807 19:48:43.977207 8520 tablet_bootstrap.cc:492] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be: No bootstrap required, opened a new log
I20260807 19:48:43.977394 8310 raft_consensus.cc:2468] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 0.
I20260807 19:48:43.977445 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
I20260807 19:48:43.978116 8196 raft_consensus.cc:3060] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:43.980075 8446 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27887cbd32644b5286ac76a1a25f550f" is_pre_election: true
I20260807 19:48:43.980957 8446 raft_consensus.cc:2393] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e287c74a50ec40ecadff065dd8be7c5a in current term 1: Already voted for candidate 27887cbd32644b5286ac76a1a25f550f in this term.
I20260807 19:48:43.979558 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:43.982486 8310 raft_consensus.cc:3060] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:43.983178 8140 leader_election.cc:343] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:43.983780 8450 raft_consensus.cc:2468] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 1.
I20260807 19:48:43.983984 8140 leader_election.cc:304] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c2790d840c74645a489a5918a9ad5be, e287c74a50ec40ecadff065dd8be7c5a; no voters: 27887cbd32644b5286ac76a1a25f550f
I20260807 19:48:43.985396 8196 raft_consensus.cc:2468] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 1.
I20260807 19:48:43.985822 8140 leader_election.cc:304] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters: 3c2790d840c74645a489a5918a9ad5be
I20260807 19:48:43.977675 8520 ts_tablet_manager.cc:1403] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.007s
I20260807 19:48:43.985999 8561 raft_consensus.cc:2764] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 19:48:43.986966 8561 raft_consensus.cc:2804] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:43.987442 8561 raft_consensus.cc:697] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 1 LEADER]: Becoming Leader. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Running, Role: LEADER
I20260807 19:48:43.988281 8561 consensus_queue.cc:237] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:43.988901 8520 raft_consensus.cc:359] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.989524 8520 raft_consensus.cc:385] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:43.989749 8520 raft_consensus.cc:740] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2790d840c74645a489a5918a9ad5be, State: Initialized, Role: FOLLOWER
I20260807 19:48:43.989944 8521 leader_election.cc:290] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Requested vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:43.990549 8520 consensus_queue.cc:260] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:43.991463 8310 raft_consensus.cc:2468] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 1.
I20260807 19:48:43.992314 8520 ts_tablet_manager.cc:1434] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 19:48:43.992686 8379 leader_election.cc:304] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, 3c2790d840c74645a489a5918a9ad5be; no voters:
I20260807 19:48:43.993433 8520 tablet_bootstrap.cc:492] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be: Bootstrap starting.
I20260807 19:48:43.996440 8521 raft_consensus.cc:2804] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:43.997140 8521 raft_consensus.cc:697] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Becoming Leader. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Running, Role: LEADER
I20260807 19:48:43.998086 8521 consensus_queue.cc:237] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.000052 8520 tablet_bootstrap.cc:654] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:44.014410 8520 tablet_bootstrap.cc:492] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be: No bootstrap required, opened a new log
I20260807 19:48:44.014943 8520 ts_tablet_manager.cc:1403] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.013s
I20260807 19:48:44.015318 8030 catalog_manager.cc:5697] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193). New cstate: current_term: 1 leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:44.016861 8029 catalog_manager.cc:5697] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f reported cstate change: term changed from 0 to 1, leader changed from <none> to 27887cbd32644b5286ac76a1a25f550f (127.6.42.195). New cstate: current_term: 1 leader_uuid: "27887cbd32644b5286ac76a1a25f550f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:44.018028 8520 raft_consensus.cc:359] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.018885 8520 raft_consensus.cc:385] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:44.019222 8520 raft_consensus.cc:740] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2790d840c74645a489a5918a9ad5be, State: Initialized, Role: FOLLOWER
I20260807 19:48:44.020016 8520 consensus_queue.cc:260] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.021862 8520 ts_tablet_manager.cc:1434] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 19:48:44.031369 8532 raft_consensus.cc:493] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:44.031930 8532 raft_consensus.cc:515] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.033272 8521 raft_consensus.cc:493] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:44.033694 8521 raft_consensus.cc:515] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.034770 8532 leader_election.cc:290] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:44.035616 8521 leader_election.cc:290] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:44.037909 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b921e5ba58f841808417b01e7f5ac497" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a" is_pre_election: true
I20260807 19:48:44.038626 8196 raft_consensus.cc:2468] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 0.
I20260807 19:48:44.039788 8382 leader_election.cc:304] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:44.039999 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b921e5ba58f841808417b01e7f5ac497" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
I20260807 19:48:44.040688 8310 raft_consensus.cc:2468] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 0.
I20260807 19:48:44.041502 8521 raft_consensus.cc:2804] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:44.041898 8521 raft_consensus.cc:493] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:44.042225 8521 raft_consensus.cc:3060] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.047461 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254a1cd9838545e58769e0f60d8ef4f5" candidate_uuid: "3c2790d840c74645a489a5918a9ad5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a" is_pre_election: true
I20260807 19:48:44.049255 8521 raft_consensus.cc:515] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.050446 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "254a1cd9838545e58769e0f60d8ef4f5" candidate_uuid: "3c2790d840c74645a489a5918a9ad5be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27887cbd32644b5286ac76a1a25f550f" is_pre_election: true
I20260807 19:48:44.051090 8450 raft_consensus.cc:2393] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c2790d840c74645a489a5918a9ad5be in current term 1: Already voted for candidate e287c74a50ec40ecadff065dd8be7c5a in this term.
I20260807 19:48:44.051072 8521 leader_election.cc:290] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Requested vote from peers e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:44.051889 8196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b921e5ba58f841808417b01e7f5ac497" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
I20260807 19:48:44.052232 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b921e5ba58f841808417b01e7f5ac497" candidate_uuid: "27887cbd32644b5286ac76a1a25f550f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:44.052382 8196 raft_consensus.cc:3060] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.052479 8561 raft_consensus.cc:493] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:44.052908 8310 raft_consensus.cc:3060] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.052929 8258 leader_election.cc:304] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c2790d840c74645a489a5918a9ad5be; no voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a
I20260807 19:48:44.053194 8561 raft_consensus.cc:515] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.053901 8532 raft_consensus.cc:3060] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.055739 8561 leader_election.cc:290] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:44.057040 8311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f42a2bc8ddd4ec4aebf3e1c1e892cd6" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be" is_pre_election: true
I20260807 19:48:44.057770 8311 raft_consensus.cc:2468] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 0.
I20260807 19:48:44.056972 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f42a2bc8ddd4ec4aebf3e1c1e892cd6" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27887cbd32644b5286ac76a1a25f550f" is_pre_election: true
I20260807 19:48:44.059219 8140 leader_election.cc:304] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c2790d840c74645a489a5918a9ad5be, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:44.059640 8450 raft_consensus.cc:2468] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 0.
I20260807 19:48:44.060117 8561 raft_consensus.cc:2804] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:44.060503 8561 raft_consensus.cc:493] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:44.060851 8561 raft_consensus.cc:3060] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.061604 8532 raft_consensus.cc:2749] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:44.061859 8196 raft_consensus.cc:2468] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 1.
I20260807 19:48:44.063145 8382 leader_election.cc:304] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:44.063890 8310 raft_consensus.cc:2468] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27887cbd32644b5286ac76a1a25f550f in term 1.
I20260807 19:48:44.064002 8521 raft_consensus.cc:2804] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:44.064710 8521 raft_consensus.cc:697] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Becoming Leader. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Running, Role: LEADER
I20260807 19:48:44.065608 8521 consensus_queue.cc:237] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.068197 8561 raft_consensus.cc:515] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.070441 8561 leader_election.cc:290] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 election: Requested vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:44.071534 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f42a2bc8ddd4ec4aebf3e1c1e892cd6" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
I20260807 19:48:44.071621 8310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f42a2bc8ddd4ec4aebf3e1c1e892cd6" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:44.072024 8450 raft_consensus.cc:3060] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.072183 8310 raft_consensus.cc:3060] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.076100 8032 catalog_manager.cc:5697] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f reported cstate change: term changed from 0 to 1, leader changed from <none> to 27887cbd32644b5286ac76a1a25f550f (127.6.42.195). New cstate: current_term: 1 leader_uuid: "27887cbd32644b5286ac76a1a25f550f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:44.077530 8450 raft_consensus.cc:2468] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 1.
I20260807 19:48:44.078506 8140 leader_election.cc:304] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:44.079069 8310 raft_consensus.cc:2468] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 1.
I20260807 19:48:44.079169 8561 raft_consensus.cc:2804] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:44.082032 8561 raft_consensus.cc:697] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 1 LEADER]: Becoming Leader. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Running, Role: LEADER
I20260807 19:48:44.082859 8561 consensus_queue.cc:237] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.090891 8032 catalog_manager.cc:5697] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193). New cstate: current_term: 1 leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:44.236729 8450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557"
dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
mode: GRACEFUL
new_leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
from {username='slave'} at 127.0.0.1:47942
I20260807 19:48:44.237303 8450 raft_consensus.cc:606] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Received request to transfer leadership to TS e287c74a50ec40ecadff065dd8be7c5a
I20260807 19:48:44.243000 8450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a"
dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
mode: GRACEFUL
new_leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
from {username='slave'} at 127.0.0.1:47942
I20260807 19:48:44.243544 8450 raft_consensus.cc:606] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Received request to transfer leadership to TS e287c74a50ec40ecadff065dd8be7c5a
I20260807 19:48:44.248821 8450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b921e5ba58f841808417b01e7f5ac497"
dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
mode: GRACEFUL
new_leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
from {username='slave'} at 127.0.0.1:47942
I20260807 19:48:44.249401 8450 raft_consensus.cc:606] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Received request to transfer leadership to TS e287c74a50ec40ecadff065dd8be7c5a
I20260807 19:48:44.254959 8450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e0d11c546d94f48b9eb32a0d6d74652"
dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
mode: GRACEFUL
new_leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
from {username='slave'} at 127.0.0.1:47942
I20260807 19:48:44.255497 8450 raft_consensus.cc:606] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Received request to transfer leadership to TS e287c74a50ec40ecadff065dd8be7c5a
I20260807 19:48:44.276924 8521 consensus_queue.cc:1048] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:44.284986 8521 consensus_queue.cc:1048] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:44.286347 8539 raft_consensus.cc:993] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: : Instructing follower e287c74a50ec40ecadff065dd8be7c5a to start an election
I20260807 19:48:44.286795 8539 raft_consensus.cc:1081] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Signalling peer e287c74a50ec40ecadff065dd8be7c5a to start an election
I20260807 19:48:44.290127 8190 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a"
dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
from {username='slave'} at 127.0.0.1:57838
I20260807 19:48:44.291779 8190 raft_consensus.cc:493] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:44.292201 8190 raft_consensus.cc:3060] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:44.296790 8310 raft_consensus.cc:3060] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.301005 8190 raft_consensus.cc:515] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.306089 8190 leader_election.cc:290] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 2 election: Requested vote from peers 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513), 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985)
I20260807 19:48:44.306464 8534 consensus_queue.cc:1048] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:44.307868 8311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:44.308087 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44877db0c4c54c1d9ee4a50184c4e63a" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
I20260807 19:48:44.308570 8311 raft_consensus.cc:3060] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:44.310276 8450 raft_consensus.cc:3055] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:44.310739 8450 raft_consensus.cc:740] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Running, Role: LEADER
I20260807 19:48:44.311615 8450 consensus_queue.cc:260] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.312870 8450 raft_consensus.cc:3060] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:44.316383 8545 raft_consensus.cc:993] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: : Instructing follower e287c74a50ec40ecadff065dd8be7c5a to start an election
I20260807 19:48:44.316927 8545 raft_consensus.cc:1081] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Signalling peer e287c74a50ec40ecadff065dd8be7c5a to start an election
I20260807 19:48:44.319011 8190 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557"
dest_uuid: "e287c74a50ec40ecadff065dd8be7c5a"
from {username='slave'} at 127.0.0.1:57838
I20260807 19:48:44.319607 8190 raft_consensus.cc:493] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:48:44.320035 8190 raft_consensus.cc:3060] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:44.324404 8311 raft_consensus.cc:2468] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 2.
I20260807 19:48:44.325775 8313 raft_consensus.cc:1240] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Rejecting Update request from peer 27887cbd32644b5286ac76a1a25f550f for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 19:48:44.326929 8140 leader_election.cc:304] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c2790d840c74645a489a5918a9ad5be, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:44.327592 8450 raft_consensus.cc:2468] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 2.
I20260807 19:48:44.329809 8310 raft_consensus.cc:3060] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.331131 8561 raft_consensus.cc:2804] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:44.335366 8561 raft_consensus.cc:697] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 2 LEADER]: Becoming Leader. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Running, Role: LEADER
I20260807 19:48:44.332316 8190 raft_consensus.cc:515] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:44.336552 8561 consensus_queue.cc:237] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } }
I20260807 19:48:44.340649 8190 leader_election.cc:290] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 2 election: Requested vote from peers 27887cbd32644b5286ac76a1a25f550f (127.6.42.195:37985), 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513)
I20260807 19:48:44.340914 8450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "27887cbd32644b5286ac76a1a25f550f"
I20260807 19:48:44.341553 8450 raft_consensus.cc:3055] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:44.341910 8450 raft_consensus.cc:740] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 27887cbd32644b5286ac76a1a25f550f, State: Running, Role: LEADER
I20260807 19:48:44.342900 8450 consensus_queue.cc:260] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:44.343756 8313 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ea32be48a34c68878ed7f40aa64557" candidate_uuid: "e287c74a50ec40ecadff065dd8be7c5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3c2790d840c74645a489a5918a9ad5be"
I20260807 19:48:44.344441 8313 raft_consensus.cc:3060] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:44.346225 8545 consensus_queue.cc:1048] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:44.349586 8450 raft_consensus.cc:3060] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:44.349543 8032 catalog_manager.cc:5697] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a reported cstate change: term changed from 1 to 2, leader changed from 27887cbd32644b5286ac76a1a25f550f (127.6.42.195) to e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193). New cstate: current_term: 2 leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:44.352041 8313 raft_consensus.cc:2468] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 2.
I20260807 19:48:44.357769 8450 raft_consensus.cc:2468] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e287c74a50ec40ecadff065dd8be7c5a in term 2.
I20260807 19:48:44.363731 8140 leader_election.cc:304] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27887cbd32644b5286ac76a1a25f550f, e287c74a50ec40ecadff065dd8be7c5a; no voters:
I20260807 19:48:44.364763 8561 raft_consensus.cc:2804] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:44.365315 8561 raft_consensus.cc:697] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 2 LEADER]: Becoming Leader. State: Replica: e287c74a50ec40ecadff065dd8be7c5a, State: Running, Role: LEADER
I20260807 19:48:44.366202 8561 consensus_queue.cc:237] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } }
I20260807 19:48:44.374374 8032 catalog_manager.cc:5697] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a reported cstate change: term changed from 1 to 2, leader changed from 27887cbd32644b5286ac76a1a25f550f (127.6.42.195) to e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193). New cstate: current_term: 2 leader_uuid: "e287c74a50ec40ecadff065dd8be7c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e287c74a50ec40ecadff065dd8be7c5a" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 42663 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:44.377449 8561 consensus_queue.cc:1048] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:44.394013 8561 consensus_queue.cc:1048] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "27887cbd32644b5286ac76a1a25f550f" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 37985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:44.410677 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 19:48:44.415498 6315 auto_leader_rebalancer.cc:456] leader transfer for tablet 44877db0c4c54c1d9ee4a50184c4e63a from e287c74a50ec40ecadff065dd8be7c5a to 3c2790d840c74645a489a5918a9ad5be failed: Service unavailable: TEST: forced leader step-down failure
W20260807 19:48:44.416003 6315 auto_leader_rebalancer.cc:456] leader transfer for tablet 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 from e287c74a50ec40ecadff065dd8be7c5a to 3c2790d840c74645a489a5918a9ad5be failed: Service unavailable: TEST: forced leader step-down failure
I20260807 19:48:44.416184 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260807 19:48:44.416509 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:48:44.416895 6315 tablet_server.cc:179] TabletServer@127.6.42.193:0 shutting down...
I20260807 19:48:44.434676 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:44.435321 6315 tablet_replica.cc:333] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a: stopping tablet replica
I20260807 19:48:44.436038 6315 raft_consensus.cc:2243] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.436560 6315 raft_consensus.cc:2272] T 2e0d11c546d94f48b9eb32a0d6d74652 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.438302 6315 tablet_replica.cc:333] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a: stopping tablet replica
I20260807 19:48:44.438799 6315 raft_consensus.cc:2243] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.439538 6315 raft_consensus.cc:2272] T 254a1cd9838545e58769e0f60d8ef4f5 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.441758 6315 tablet_replica.cc:333] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a: stopping tablet replica
I20260807 19:48:44.442229 6315 raft_consensus.cc:2243] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.442637 6315 raft_consensus.cc:2272] T b921e5ba58f841808417b01e7f5ac497 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.444181 6315 tablet_replica.cc:333] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: stopping tablet replica
I20260807 19:48:44.444680 6315 raft_consensus.cc:2243] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.445199 6315 pending_rounds.cc:70] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: Trying to abort 1 pending ops.
I20260807 19:48:44.445397 6315 pending_rounds.cc:77] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315997181416177664 op_type: NO_OP noop_request { }
I20260807 19:48:44.445685 6315 raft_consensus.cc:2889] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 19:48:44.445919 6315 raft_consensus.cc:2272] T c5ea32be48a34c68878ed7f40aa64557 P e287c74a50ec40ecadff065dd8be7c5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.447901 6315 tablet_replica.cc:333] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: stopping tablet replica
I20260807 19:48:44.448334 6315 raft_consensus.cc:2243] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.448923 6315 pending_rounds.cc:70] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: Trying to abort 1 pending ops.
I20260807 19:48:44.449070 6315 pending_rounds.cc:77] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315997180251713536 op_type: NO_OP noop_request { }
I20260807 19:48:44.449288 6315 raft_consensus.cc:2889] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 19:48:44.449496 6315 raft_consensus.cc:2272] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P e287c74a50ec40ecadff065dd8be7c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.451404 6315 tablet_replica.cc:333] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: stopping tablet replica
I20260807 19:48:44.451813 6315 raft_consensus.cc:2243] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 2 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.452315 6315 pending_rounds.cc:70] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: Trying to abort 1 pending ops.
I20260807 19:48:44.452539 6315 pending_rounds.cc:77] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7315997181293973504 op_type: NO_OP noop_request { }
I20260807 19:48:44.452775 6315 raft_consensus.cc:2889] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 19:48:44.452997 6315 raft_consensus.cc:2272] T 44877db0c4c54c1d9ee4a50184c4e63a P e287c74a50ec40ecadff065dd8be7c5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.477015 6315 tablet_server.cc:196] TabletServer@127.6.42.193:0 shutdown complete.
I20260807 19:48:44.485774 8545 consensus_queue.cc:1048] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c2790d840c74645a489a5918a9ad5be" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 34513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:44.490832 6315 tablet_server.cc:179] TabletServer@127.6.42.194:0 shutting down...
W20260807 19:48:44.515717 8379 consensus_peers.cc:597] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f -> Peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Couldn't send request to peer 3c2790d840c74645a489a5918a9ad5be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260807 19:48:44.518930 8382 consensus_peers.cc:597] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f -> Peer e287c74a50ec40ecadff065dd8be7c5a (127.6.42.193:42663): Couldn't send request to peer e287c74a50ec40ecadff065dd8be7c5a. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.193:42663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:44.524830 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:44.525525 6315 tablet_replica.cc:333] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be: stopping tablet replica
I20260807 19:48:44.526120 6315 raft_consensus.cc:2243] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.526538 6315 raft_consensus.cc:2272] T c5ea32be48a34c68878ed7f40aa64557 P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.528517 6315 tablet_replica.cc:333] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: stopping tablet replica
I20260807 19:48:44.529054 6315 raft_consensus.cc:2243] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.529423 6315 pending_rounds.cc:70] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: Trying to abort 1 pending ops.
I20260807 19:48:44.529585 6315 pending_rounds.cc:77] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315997180182544384 op_type: NO_OP noop_request { }
I20260807 19:48:44.529863 6315 raft_consensus.cc:2889] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260807 19:48:44.530121 6315 raft_consensus.cc:2272] T b921e5ba58f841808417b01e7f5ac497 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.532285 6315 tablet_replica.cc:333] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be: stopping tablet replica
I20260807 19:48:44.532869 6315 raft_consensus.cc:2243] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.533232 6315 raft_consensus.cc:2272] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.534802 6315 tablet_replica.cc:333] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be: stopping tablet replica
I20260807 19:48:44.535288 6315 raft_consensus.cc:2243] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.535640 6315 raft_consensus.cc:2272] T 44877db0c4c54c1d9ee4a50184c4e63a P 3c2790d840c74645a489a5918a9ad5be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.537189 6315 tablet_replica.cc:333] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be: stopping tablet replica
I20260807 19:48:44.537637 6315 raft_consensus.cc:2243] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.538014 6315 raft_consensus.cc:2272] T 254a1cd9838545e58769e0f60d8ef4f5 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.540042 6315 tablet_replica.cc:333] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be: stopping tablet replica
I20260807 19:48:44.540572 6315 raft_consensus.cc:2243] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.540946 6315 raft_consensus.cc:2272] T 2e0d11c546d94f48b9eb32a0d6d74652 P 3c2790d840c74645a489a5918a9ad5be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.564327 6315 tablet_server.cc:196] TabletServer@127.6.42.194:0 shutdown complete.
W20260807 19:48:44.571700 8379 consensus_peers.cc:597] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f -> Peer 3c2790d840c74645a489a5918a9ad5be (127.6.42.194:34513): Couldn't send request to peer 3c2790d840c74645a489a5918a9ad5be. Status: Network error: Client connection negotiation failed: client connection to 127.6.42.194:34513: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:44.577785 6315 tablet_server.cc:179] TabletServer@127.6.42.195:0 shutting down...
I20260807 19:48:44.596557 6315 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260807 19:48:44.597278 6315 tablet_replica.cc:333] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f: stopping tablet replica
I20260807 19:48:44.598037 6315 raft_consensus.cc:2243] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.598575 6315 raft_consensus.cc:2272] T 254a1cd9838545e58769e0f60d8ef4f5 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.600867 6315 tablet_replica.cc:333] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: stopping tablet replica
I20260807 19:48:44.601401 6315 raft_consensus.cc:2243] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.602054 6315 pending_rounds.cc:70] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: Trying to abort 1 pending ops.
I20260807 19:48:44.602247 6315 pending_rounds.cc:77] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315997179906879488 op_type: NO_OP noop_request { }
I20260807 19:48:44.602507 6315 raft_consensus.cc:2889] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260807 19:48:44.602777 6315 raft_consensus.cc:2272] T 2e0d11c546d94f48b9eb32a0d6d74652 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.605029 6315 tablet_replica.cc:333] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f: stopping tablet replica
I20260807 19:48:44.605528 6315 raft_consensus.cc:2243] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.605904 6315 raft_consensus.cc:2272] T 4f42a2bc8ddd4ec4aebf3e1c1e892cd6 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.607445 6315 tablet_replica.cc:333] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f: stopping tablet replica
I20260807 19:48:44.607923 6315 raft_consensus.cc:2243] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.608319 6315 raft_consensus.cc:2272] T 44877db0c4c54c1d9ee4a50184c4e63a P 27887cbd32644b5286ac76a1a25f550f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.610397 6315 tablet_replica.cc:333] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f: stopping tablet replica
I20260807 19:48:44.610886 6315 raft_consensus.cc:2243] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.611637 6315 raft_consensus.cc:2272] T b921e5ba58f841808417b01e7f5ac497 P 27887cbd32644b5286ac76a1a25f550f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.613711 6315 tablet_replica.cc:333] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f: stopping tablet replica
I20260807 19:48:44.614178 6315 raft_consensus.cc:2243] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:44.614593 6315 raft_consensus.cc:2272] T c5ea32be48a34c68878ed7f40aa64557 P 27887cbd32644b5286ac76a1a25f550f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.639101 6315 tablet_server.cc:196] TabletServer@127.6.42.195:0 shutdown complete.
I20260807 19:48:44.651934 6315 master.cc:562] Master@127.6.42.254:35721 shutting down...
I20260807 19:48:44.667531 6315 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:48:44.668080 6315 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:44.668388 6315 tablet_replica.cc:333] T 00000000000000000000000000000000 P 20fc144bd2764ac283bb04a7da62d2ce: stopping tablet replica
I20260807 19:48:44.678696 6315 master.cc:584] Master@127.6.42.254:35721 shutdown complete.
[ OK ] LeaderRebalancerTest.TestStepDownFailureDoesNotAbortPass (1894 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260807 19:48:44.706496 6315 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.42.254:41447
I20260807 19:48:44.707617 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:44.712710 8600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:44.713160 8601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:44.715137 6315 server_base.cc:1061] running on GCE node
W20260807 19:48:44.715090 8603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:44.716156 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:44.716382 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:44.716547 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132124716526 us; error 0 us; skew 500 ppm
I20260807 19:48:44.717096 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:44.719360 6315 webserver.cc:533] Webserver started at http://127.6.42.254:34149/ using document root <none> and password file <none>
I20260807 19:48:44.719867 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:44.720049 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:44.720332 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:44.721520 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/master-0-root/instance:
uuid: "9ee08994477a4e94a231c734242ac45f"
format_stamp: "Formatted at 2026-08-07 19:48:44 on dist-test-slave-lkx2"
I20260807 19:48:44.726239 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:48:44.729903 8611 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:44.730705 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 19:48:44.730985 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/master-0-root
uuid: "9ee08994477a4e94a231c734242ac45f"
format_stamp: "Formatted at 2026-08-07 19:48:44 on dist-test-slave-lkx2"
I20260807 19:48:44.731244 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:44.742048 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:44.743114 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:44.780333 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.254:41447
I20260807 19:48:44.780458 8695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.254:41447 every 8 connection(s)
I20260807 19:48:44.784106 8696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:44.794351 8696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Bootstrap starting.
I20260807 19:48:44.798754 8696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:44.803006 8696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: No bootstrap required, opened a new log
I20260807 19:48:44.805136 8696 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee08994477a4e94a231c734242ac45f" member_type: VOTER }
I20260807 19:48:44.805524 8696 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:44.805765 8696 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee08994477a4e94a231c734242ac45f, State: Initialized, Role: FOLLOWER
I20260807 19:48:44.806335 8696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee08994477a4e94a231c734242ac45f" member_type: VOTER }
I20260807 19:48:44.806787 8696 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:48:44.807042 8696 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:48:44.807294 8696 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:44.811904 8696 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee08994477a4e94a231c734242ac45f" member_type: VOTER }
I20260807 19:48:44.812500 8696 leader_election.cc:304] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ee08994477a4e94a231c734242ac45f; no voters:
I20260807 19:48:44.813661 8696 leader_election.cc:290] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:48:44.813980 8701 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:44.815340 8701 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [term 1 LEADER]: Becoming Leader. State: Replica: 9ee08994477a4e94a231c734242ac45f, State: Running, Role: LEADER
I20260807 19:48:44.816025 8701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee08994477a4e94a231c734242ac45f" member_type: VOTER }
I20260807 19:48:44.816555 8696 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:48:44.819032 8702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ee08994477a4e94a231c734242ac45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee08994477a4e94a231c734242ac45f" member_type: VOTER } }
I20260807 19:48:44.819103 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ee08994477a4e94a231c734242ac45f. Latest consensus state: current_term: 1 leader_uuid: "9ee08994477a4e94a231c734242ac45f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ee08994477a4e94a231c734242ac45f" member_type: VOTER } }
I20260807 19:48:44.819931 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:44.819952 8702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f [sys.catalog]: This master's current role is: LEADER
I20260807 19:48:44.825757 8706 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260807 19:48:44.831095 8706 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260807 19:48:44.832588 6315 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260807 19:48:44.840431 8706 catalog_manager.cc:1383] Generated new cluster ID: 53754977221b42e2b571accb7d64033e
I20260807 19:48:44.840764 8706 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260807 19:48:44.857456 8706 catalog_manager.cc:1406] Generated new certificate authority record
I20260807 19:48:44.858807 8706 catalog_manager.cc:1540] Loading token signing keys...
I20260807 19:48:44.882316 8706 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Generated new TSK 0
I20260807 19:48:44.882978 8706 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260807 19:48:44.899698 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:44.905668 8724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:44.906548 8725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:44.908761 8727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:44.908978 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:44.910252 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:44.910435 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:44.910558 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132124910547 us; error 0 us; skew 500 ppm
I20260807 19:48:44.911051 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:44.913484 6315 webserver.cc:533] Webserver started at http://127.6.42.193:38089/ using document root <none> and password file <none>
I20260807 19:48:44.913946 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:44.914103 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:44.914315 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:44.915339 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/instance:
uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
format_stamp: "Formatted at 2026-08-07 19:48:44 on dist-test-slave-lkx2"
I20260807 19:48:44.920217 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 19:48:44.923575 8735 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:44.924315 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 19:48:44.924628 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root
uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
format_stamp: "Formatted at 2026-08-07 19:48:44 on dist-test-slave-lkx2"
I20260807 19:48:44.924933 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:44.944868 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:44.946080 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:44.947610 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:44.950117 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:44.950366 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:44.950605 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:44.950774 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:44.989952 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.193:33185
I20260807 19:48:44.990080 8828 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.193:33185 every 8 connection(s)
I20260807 19:48:44.994807 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:45.001518 8836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:45.002225 8837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:45.007337 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:45.008119 8831 heartbeater.cc:344] Connected to a master server at 127.6.42.254:41447
I20260807 19:48:45.008512 8831 heartbeater.cc:461] Registering TS with master...
W20260807 19:48:45.009101 8839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:45.009230 8831 heartbeater.cc:507] Master 127.6.42.254:41447 requested a full tablet report, sending...
I20260807 19:48:45.010167 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:45.010396 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:45.010561 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132125010544 us; error 0 us; skew 500 ppm
I20260807 19:48:45.011242 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:45.011857 8632 ts_manager.cc:194] Registered new tserver with Master: 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
I20260807 19:48:45.014219 6315 webserver.cc:533] Webserver started at http://127.6.42.194:45867/ using document root <none> and password file <none>
I20260807 19:48:45.014338 8632 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38426
I20260807 19:48:45.014964 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:45.015240 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:45.015561 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:45.017050 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root/instance:
uuid: "671d8754170646eaac5748745eba58b4"
format_stamp: "Formatted at 2026-08-07 19:48:45 on dist-test-slave-lkx2"
I20260807 19:48:45.021929 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:48:45.025633 8848 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:45.026531 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:48:45.027109 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root
uuid: "671d8754170646eaac5748745eba58b4"
format_stamp: "Formatted at 2026-08-07 19:48:45 on dist-test-slave-lkx2"
I20260807 19:48:45.027511 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:45.046469 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:45.047624 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:45.049129 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:45.051420 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:45.051620 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:45.051851 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:45.052003 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:45.090699 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.194:37659
I20260807 19:48:45.090807 8951 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.194:37659 every 8 connection(s)
I20260807 19:48:45.095444 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:45.102267 8957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:45.103713 8958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:45.104926 8961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:45.107275 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:45.108049 8952 heartbeater.cc:344] Connected to a master server at 127.6.42.254:41447
I20260807 19:48:45.108296 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260807 19:48:45.108450 8952 heartbeater.cc:461] Registering TS with master...
W20260807 19:48:45.108608 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:45.108839 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132125108821 us; error 0 us; skew 500 ppm
I20260807 19:48:45.109316 8952 heartbeater.cc:507] Master 127.6.42.254:41447 requested a full tablet report, sending...
I20260807 19:48:45.109414 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:45.111573 8632 ts_manager.cc:194] Registered new tserver with Master: 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:45.112104 6315 webserver.cc:533] Webserver started at http://127.6.42.195:34627/ using document root <none> and password file <none>
I20260807 19:48:45.112650 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:45.112865 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:45.113119 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:45.113247 8632 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38434
I20260807 19:48:45.114238 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root/instance:
uuid: "a38f4d6088cf4c8cb510200ca0555801"
format_stamp: "Formatted at 2026-08-07 19:48:45 on dist-test-slave-lkx2"
I20260807 19:48:45.119241 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:48:45.122712 8967 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:45.123687 6315 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 19:48:45.124011 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root
uuid: "a38f4d6088cf4c8cb510200ca0555801"
format_stamp: "Formatted at 2026-08-07 19:48:45 on dist-test-slave-lkx2"
I20260807 19:48:45.124294 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:45.137270 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:45.138402 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:45.139878 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:45.142205 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:45.142395 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:45.142650 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:45.142795 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:45.185035 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.195:35801
I20260807 19:48:45.185143 9068 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.195:35801 every 8 connection(s)
I20260807 19:48:45.199617 9070 heartbeater.cc:344] Connected to a master server at 127.6.42.254:41447
I20260807 19:48:45.200066 9070 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:45.200965 9070 heartbeater.cc:507] Master 127.6.42.254:41447 requested a full tablet report, sending...
I20260807 19:48:45.202926 8632 ts_manager.cc:194] Registered new tserver with Master: a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:45.203351 6315 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014952177s
I20260807 19:48:45.204285 8632 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38436
I20260807 19:48:45.229368 8632 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38440:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:48:45.235421 8632 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:48:45.457954 8774 tablet_service.cc:1511] Processing CreateTablet for tablet d8160be756c34002a2ba00cf8deaa10a (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260807 19:48:45.459653 8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8160be756c34002a2ba00cf8deaa10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.460075 8771 tablet_service.cc:1511] Processing CreateTablet for tablet ea5ba8c8385a404799a0be85834f98f4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260807 19:48:45.461427 8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea5ba8c8385a404799a0be85834f98f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.463191 8898 tablet_service.cc:1511] Processing CreateTablet for tablet ea5ba8c8385a404799a0be85834f98f4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260807 19:48:45.464442 8773 tablet_service.cc:1511] Processing CreateTablet for tablet 06dc193f54374639a09c9b5827ff15f4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260807 19:48:45.465404 8894 tablet_service.cc:1511] Processing CreateTablet for tablet 413683c7378447239874762d812d75e5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260807 19:48:45.466792 8894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413683c7378447239874762d812d75e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.466497 8900 tablet_service.cc:1511] Processing CreateTablet for tablet d8160be756c34002a2ba00cf8deaa10a (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260807 19:48:45.467790 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8160be756c34002a2ba00cf8deaa10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.467367 8770 tablet_service.cc:1511] Processing CreateTablet for tablet 413683c7378447239874762d812d75e5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260807 19:48:45.469077 8893 tablet_service.cc:1511] Processing CreateTablet for tablet e10d78176f8545489a4700b7bd08fec3 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260807 19:48:45.469650 8765 tablet_service.cc:1511] Processing CreateTablet for tablet fced846726734919ac0881f9a26379ba (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260807 19:48:45.470752 8768 tablet_service.cc:1511] Processing CreateTablet for tablet d54b306954094de7a87c17dbbf6ef21f (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260807 19:48:45.470852 8889 tablet_service.cc:1511] Processing CreateTablet for tablet d54b306954094de7a87c17dbbf6ef21f (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260807 19:48:45.465405 8899 tablet_service.cc:1511] Processing CreateTablet for tablet 06dc193f54374639a09c9b5827ff15f4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260807 19:48:45.472932 8764 tablet_service.cc:1511] Processing CreateTablet for tablet 7572f4807051425c936e926fa8c8d74e (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260807 19:48:45.473460 8888 tablet_service.cc:1511] Processing CreateTablet for tablet 537cbacddca2410db509aff60875a134 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260807 19:48:45.474665 8881 tablet_service.cc:1511] Processing CreateTablet for tablet 7572f4807051425c936e926fa8c8d74e (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260807 19:48:45.469676 8769 tablet_service.cc:1511] Processing CreateTablet for tablet e10d78176f8545489a4700b7bd08fec3 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260807 19:48:45.475780 8885 tablet_service.cc:1511] Processing CreateTablet for tablet 7e8d547d81964989bb6cd1b677c971c4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260807 19:48:45.474014 8766 tablet_service.cc:1511] Processing CreateTablet for tablet 7e8d547d81964989bb6cd1b677c971c4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260807 19:48:45.478011 8884 tablet_service.cc:1511] Processing CreateTablet for tablet fced846726734919ac0881f9a26379ba (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260807 19:48:45.472232 8889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d54b306954094de7a87c17dbbf6ef21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.472932 8767 tablet_service.cc:1511] Processing CreateTablet for tablet 537cbacddca2410db509aff60875a134 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260807 19:48:45.482585 8899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06dc193f54374639a09c9b5827ff15f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.465831 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06dc193f54374639a09c9b5827ff15f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.484874 8898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea5ba8c8385a404799a0be85834f98f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.486121 8768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d54b306954094de7a87c17dbbf6ef21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.488511 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 537cbacddca2410db509aff60875a134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.492795 8769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e10d78176f8545489a4700b7bd08fec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.496816 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8d547d81964989bb6cd1b677c971c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.501250 8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7572f4807051425c936e926fa8c8d74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.503800 9101 tablet_bootstrap.cc:492] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:45.504498 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413683c7378447239874762d812d75e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.505069 8771 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe2de08a4ba49eea927f371b4bf0e5f (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260807 19:48:45.506625 8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe2de08a4ba49eea927f371b4bf0e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.510985 8764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7572f4807051425c936e926fa8c8d74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.506839 8900 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe2de08a4ba49eea927f371b4bf0e5f (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260807 19:48:45.512110 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe2de08a4ba49eea927f371b4bf0e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.513127 9101 tablet_bootstrap.cc:654] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:45.515332 8767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 537cbacddca2410db509aff60875a134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.517591 9099 tablet_bootstrap.cc:492] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:45.520779 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8d547d81964989bb6cd1b677c971c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.525338 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fced846726734919ac0881f9a26379ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.525707 9006 tablet_service.cc:1511] Processing CreateTablet for tablet d8160be756c34002a2ba00cf8deaa10a (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260807 19:48:45.527163 9006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8160be756c34002a2ba00cf8deaa10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.527846 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 06dc193f54374639a09c9b5827ff15f4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260807 19:48:45.529247 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06dc193f54374639a09c9b5827ff15f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.524783 8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fced846726734919ac0881f9a26379ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.536772 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e10d78176f8545489a4700b7bd08fec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.540930 9003 tablet_service.cc:1511] Processing CreateTablet for tablet ea5ba8c8385a404799a0be85834f98f4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260807 19:48:45.541857 9099 tablet_bootstrap.cc:654] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:45.542438 9003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea5ba8c8385a404799a0be85834f98f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.551141 8900 tablet_service.cc:1511] Processing CreateTablet for tablet 65e8bf81123c4245acf858d947af51c1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260807 19:48:45.558498 9002 tablet_service.cc:1511] Processing CreateTablet for tablet 413683c7378447239874762d812d75e5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260807 19:48:45.561223 9099 tablet_bootstrap.cc:492] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:45.561115 9001 tablet_service.cc:1511] Processing CreateTablet for tablet e10d78176f8545489a4700b7bd08fec3 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260807 19:48:45.564460 9099 ts_tablet_manager.cc:1403] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.047s user 0.008s sys 0.005s
I20260807 19:48:45.564687 9002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413683c7378447239874762d812d75e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.565603 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e8bf81123c4245acf858d947af51c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.572021 8996 tablet_service.cc:1511] Processing CreateTablet for tablet 537cbacddca2410db509aff60875a134 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260807 19:48:45.563200 8773 tablet_service.cc:1511] Processing CreateTablet for tablet 2d69c395b87b43c0bf7c3d1efb63e699 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260807 19:48:45.568686 9099 raft_consensus.cc:359] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.569178 9001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e10d78176f8545489a4700b7bd08fec3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.563467 8997 tablet_service.cc:1511] Processing CreateTablet for tablet d54b306954094de7a87c17dbbf6ef21f (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260807 19:48:45.580435 9099 raft_consensus.cc:385] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:45.569700 8771 tablet_service.cc:1511] Processing CreateTablet for tablet 65e8bf81123c4245acf858d947af51c1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260807 19:48:45.580808 8997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d54b306954094de7a87c17dbbf6ef21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.581569 8996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 537cbacddca2410db509aff60875a134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.583556 9099 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:45.584852 9099 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.601179 8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e8bf81123c4245acf858d947af51c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.604949 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d69c395b87b43c0bf7c3d1efb63e699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.612668 9099 ts_tablet_manager.cc:1434] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.047s user 0.009s sys 0.001s
I20260807 19:48:45.617599 8994 tablet_service.cc:1511] Processing CreateTablet for tablet 7e8d547d81964989bb6cd1b677c971c4 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260807 19:48:45.619315 8994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8d547d81964989bb6cd1b677c971c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.631547 8770 tablet_service.cc:1511] Processing CreateTablet for tablet 133c01ab82cf4ef3b89a9649b0abf969 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260807 19:48:45.621539 8992 tablet_service.cc:1511] Processing CreateTablet for tablet fced846726734919ac0881f9a26379ba (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260807 19:48:45.633466 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fced846726734919ac0881f9a26379ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.637157 9099 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:45.642414 8831 heartbeater.cc:499] Master 127.6.42.254:41447 was elected leader, sending a full tablet report...
I20260807 19:48:45.640214 8991 tablet_service.cc:1511] Processing CreateTablet for tablet 7572f4807051425c936e926fa8c8d74e (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260807 19:48:45.652421 8991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7572f4807051425c936e926fa8c8d74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.653774 9101 tablet_bootstrap.cc:492] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:45.654316 9101 ts_tablet_manager.cc:1403] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.151s user 0.028s sys 0.036s
I20260807 19:48:45.657325 9101 raft_consensus.cc:359] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.658080 9101 raft_consensus.cc:385] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:45.658434 9101 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:45.659293 9101 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.683414 8899 tablet_service.cc:1511] Processing CreateTablet for tablet 133c01ab82cf4ef3b89a9649b0abf969 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260807 19:48:45.684773 8899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133c01ab82cf4ef3b89a9649b0abf969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.687021 8900 tablet_service.cc:1511] Processing CreateTablet for tablet 92d07404635f47998c0f61e0e7f5b732 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260807 19:48:45.688742 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d07404635f47998c0f61e0e7f5b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.691536 8888 tablet_service.cc:1511] Processing CreateTablet for tablet b8f9a221ee034d3b86101183048438ff (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260807 19:48:45.692893 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f9a221ee034d3b86101183048438ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.700953 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133c01ab82cf4ef3b89a9649b0abf969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.702402 8766 tablet_service.cc:1511] Processing CreateTablet for tablet b8f9a221ee034d3b86101183048438ff (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260807 19:48:45.705756 8764 tablet_service.cc:1511] Processing CreateTablet for tablet 92d07404635f47998c0f61e0e7f5b732 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260807 19:48:45.709406 8765 tablet_service.cc:1511] Processing CreateTablet for tablet a898110875ba40ab8e512e124aff4fb2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260807 19:48:45.710673 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a898110875ba40ab8e512e124aff4fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.719483 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 2d69c395b87b43c0bf7c3d1efb63e699 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260807 19:48:45.719945 9003 tablet_service.cc:1511] Processing CreateTablet for tablet 65e8bf81123c4245acf858d947af51c1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260807 19:48:45.721444 9001 tablet_service.cc:1511] Processing CreateTablet for tablet 133c01ab82cf4ef3b89a9649b0abf969 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260807 19:48:45.742305 9099 tablet_bootstrap.cc:654] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:45.688140 8997 tablet_service.cc:1511] Processing CreateTablet for tablet 2fe2de08a4ba49eea927f371b4bf0e5f (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
W20260807 19:48:45.744503 8832 tablet.cc:2406] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:48:45.749960 8885 tablet_service.cc:1511] Processing CreateTablet for tablet 2d69c395b87b43c0bf7c3d1efb63e699 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260807 19:48:45.751796 8764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d07404635f47998c0f61e0e7f5b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.762369 9108 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:45.763041 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f9a221ee034d3b86101183048438ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.762934 9108 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.764660 8952 heartbeater.cc:499] Master 127.6.42.254:41447 was elected leader, sending a full tablet report...
I20260807 19:48:45.765442 9106 tablet_bootstrap.cc:492] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:45.767550 9108 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:45.775013 9006 tablet_service.cc:1511] Processing CreateTablet for tablet b8f9a221ee034d3b86101183048438ff (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260807 19:48:45.784230 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d69c395b87b43c0bf7c3d1efb63e699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.795627 9101 ts_tablet_manager.cc:1434] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.141s user 0.028s sys 0.041s
I20260807 19:48:45.796721 9101 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:45.802758 9106 tablet_bootstrap.cc:654] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:45.809708 9002 tablet_service.cc:1511] Processing CreateTablet for tablet 92d07404635f47998c0f61e0e7f5b732 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260807 19:48:45.811198 9003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65e8bf81123c4245acf858d947af51c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.811887 9001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133c01ab82cf4ef3b89a9649b0abf969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.808256 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d69c395b87b43c0bf7c3d1efb63e699. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.812502 9006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8f9a221ee034d3b86101183048438ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.814421 9002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d07404635f47998c0f61e0e7f5b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.816643 8997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe2de08a4ba49eea927f371b4bf0e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.819677 9099 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:45.820209 9099 ts_tablet_manager.cc:1403] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.183s user 0.015s sys 0.000s
I20260807 19:48:45.822961 9099 raft_consensus.cc:359] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.823643 9099 raft_consensus.cc:385] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:45.823954 9099 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:45.825656 8884 tablet_service.cc:1511] Processing CreateTablet for tablet a898110875ba40ab8e512e124aff4fb2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
W20260807 19:48:45.828200 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:45.829468 8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a898110875ba40ab8e512e124aff4fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.831547 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:45.836966 8856 leader_election.cc:343] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:45.842973 9101 tablet_bootstrap.cc:654] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:45.853003 8992 tablet_service.cc:1511] Processing CreateTablet for tablet a898110875ba40ab8e512e124aff4fb2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260807 19:48:45.854699 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a898110875ba40ab8e512e124aff4fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.875501 9106 tablet_bootstrap.cc:492] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:45.876039 9106 ts_tablet_manager.cc:1403] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.111s user 0.024s sys 0.008s
I20260807 19:48:45.879177 9006 tablet_service.cc:1511] Processing CreateTablet for tablet 4ec2edf1da424177aaad8b93086118b1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260807 19:48:45.880554 9006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec2edf1da424177aaad8b93086118b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.881798 9005 tablet_service.cc:1511] Processing CreateTablet for tablet d6391ca8e3f6476d97c1983eda89cdd2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260807 19:48:45.883463 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6391ca8e3f6476d97c1983eda89cdd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.884564 9106 raft_consensus.cc:359] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.885308 9106 raft_consensus.cc:385] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:45.851497 9099 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.885615 9106 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:45.886360 8885 tablet_service.cc:1511] Processing CreateTablet for tablet 4ec2edf1da424177aaad8b93086118b1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260807 19:48:45.887477 9099 ts_tablet_manager.cc:1434] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.067s user 0.003s sys 0.004s
I20260807 19:48:45.887672 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec2edf1da424177aaad8b93086118b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.881862 8900 tablet_service.cc:1511] Processing CreateTablet for tablet d6391ca8e3f6476d97c1983eda89cdd2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260807 19:48:45.891146 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6391ca8e3f6476d97c1983eda89cdd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.889846 8766 tablet_service.cc:1511] Processing CreateTablet for tablet d6391ca8e3f6476d97c1983eda89cdd2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260807 19:48:45.889767 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:45.892158 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6391ca8e3f6476d97c1983eda89cdd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.890031 8773 tablet_service.cc:1511] Processing CreateTablet for tablet 4ec2edf1da424177aaad8b93086118b1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260807 19:48:45.893263 8849 leader_election.cc:343] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:45.896672 9106 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.897207 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:45.898278 9109 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:45.899189 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ec2edf1da424177aaad8b93086118b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.901391 8765 tablet_service.cc:1511] Processing CreateTablet for tablet e005b1fcb3134fa9a076533e372b80f9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260807 19:48:45.902797 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e005b1fcb3134fa9a076533e372b80f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.909065 8888 tablet_service.cc:1511] Processing CreateTablet for tablet e005b1fcb3134fa9a076533e372b80f9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260807 19:48:45.910764 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e005b1fcb3134fa9a076533e372b80f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.920866 9099 tablet_bootstrap.cc:492] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:45.936640 9070 heartbeater.cc:499] Master 127.6.42.254:41447 was elected leader, sending a full tablet report...
I20260807 19:48:45.942121 9107 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:45.942724 9107 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.946342 9099 tablet_bootstrap.cc:654] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:45.947717 8894 tablet_service.cc:1511] Processing CreateTablet for tablet 82ff5241853a4f548a83179766541c48 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260807 19:48:45.947746 8898 tablet_service.cc:1511] Processing CreateTablet for tablet 2e99139b6d2d4da082b4017e95030a69 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260807 19:48:45.958882 8894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ff5241853a4f548a83179766541c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.964380 8898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e99139b6d2d4da082b4017e95030a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.964320 9107 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:45.972893 9107 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:45.973325 9107 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:45.973941 9002 tablet_service.cc:1511] Processing CreateTablet for tablet 2e99139b6d2d4da082b4017e95030a69 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260807 19:48:45.975267 9002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e99139b6d2d4da082b4017e95030a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.975319 9107 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:45.981254 9003 tablet_service.cc:1511] Processing CreateTablet for tablet e005b1fcb3134fa9a076533e372b80f9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260807 19:48:45.986694 8885 tablet_service.cc:1511] Processing CreateTablet for tablet f2b65bfd5b3948cea9db5090e802132d (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260807 19:48:45.988722 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b65bfd5b3948cea9db5090e802132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.991681 9101 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:45.992558 9106 ts_tablet_manager.cc:1434] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.116s user 0.008s sys 0.012s
I20260807 19:48:45.993549 9003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e005b1fcb3134fa9a076533e372b80f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.992259 9101 ts_tablet_manager.cc:1403] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.196s user 0.024s sys 0.014s
I20260807 19:48:46.001605 8764 tablet_service.cc:1511] Processing CreateTablet for tablet 82ff5241853a4f548a83179766541c48 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260807 19:48:46.002064 8997 tablet_service.cc:1511] Processing CreateTablet for tablet f2b65bfd5b3948cea9db5090e802132d (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260807 19:48:46.003110 8764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ff5241853a4f548a83179766541c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.003412 8997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b65bfd5b3948cea9db5090e802132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:45.992592 9006 tablet_service.cc:1511] Processing CreateTablet for tablet 82ff5241853a4f548a83179766541c48 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260807 19:48:46.009104 9006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ff5241853a4f548a83179766541c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.012136 8881 tablet_service.cc:1511] Processing CreateTablet for tablet 3def68aebfd04eeab681e41691bae18c (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260807 19:48:46.012475 8770 tablet_service.cc:1511] Processing CreateTablet for tablet 2e99139b6d2d4da082b4017e95030a69 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260807 19:48:46.013726 8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3def68aebfd04eeab681e41691bae18c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.013881 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e99139b6d2d4da082b4017e95030a69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.012893 9106 tablet_bootstrap.cc:492] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:46.026439 8773 tablet_service.cc:1511] Processing CreateTablet for tablet f2b65bfd5b3948cea9db5090e802132d (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260807 19:48:46.028589 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b65bfd5b3948cea9db5090e802132d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.029492 8771 tablet_service.cc:1511] Processing CreateTablet for tablet 3def68aebfd04eeab681e41691bae18c (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260807 19:48:46.030794 8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3def68aebfd04eeab681e41691bae18c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.036458 9101 raft_consensus.cc:359] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.037305 9101 raft_consensus.cc:385] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.037616 9101 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.038537 9101 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.042325 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.043071 9101 ts_tablet_manager.cc:1434] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260807 19:48:46.051631 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.051568 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 3def68aebfd04eeab681e41691bae18c (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260807 19:48:46.052480 8915 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:46.055493 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3def68aebfd04eeab681e41691bae18c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.044186 9101 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:46.058293 9099 tablet_bootstrap.cc:492] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:46.059634 9099 ts_tablet_manager.cc:1403] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.139s user 0.012s sys 0.001s
I20260807 19:48:46.067106 9099 raft_consensus.cc:359] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.076525 9099 raft_consensus.cc:385] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.076944 9099 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.074383 9106 tablet_bootstrap.cc:654] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.080178 9099 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:46.080650 8740 leader_election.cc:343] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.081924 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:46.083487 8766 tablet_service.cc:1511] Processing CreateTablet for tablet 1ffb51072a064989a2056cfa1484f4a5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260807 19:48:46.084791 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ffb51072a064989a2056cfa1484f4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.090922 8885 tablet_service.cc:1511] Processing CreateTablet for tablet 1ffb51072a064989a2056cfa1484f4a5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260807 19:48:46.093856 9002 tablet_service.cc:1511] Processing CreateTablet for tablet 1ffb51072a064989a2056cfa1484f4a5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260807 19:48:46.095679 8770 tablet_service.cc:1511] Processing CreateTablet for tablet be89699a1aff43f1a97b936b3154c2d5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260807 19:48:46.097064 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be89699a1aff43f1a97b936b3154c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.100039 8884 tablet_service.cc:1511] Processing CreateTablet for tablet be89699a1aff43f1a97b936b3154c2d5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260807 19:48:46.100600 9117 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:46.101008 9117 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:46.101329 9117 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.101493 9101 tablet_bootstrap.cc:654] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.103641 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.104842 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.109125 9002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ffb51072a064989a2056cfa1484f4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.108785 8765 tablet_service.cc:1511] Processing CreateTablet for tablet 301b269c068f46ebae7e1c1d55e501f1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260807 19:48:46.110118 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301b269c068f46ebae7e1c1d55e501f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.103900 9099 ts_tablet_manager.cc:1434] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260807 19:48:46.111032 8881 tablet_service.cc:1511] Processing CreateTablet for tablet 301b269c068f46ebae7e1c1d55e501f1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
W20260807 19:48:46.111670 8740 leader_election.cc:343] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.112399 9005 tablet_service.cc:1511] Processing CreateTablet for tablet be89699a1aff43f1a97b936b3154c2d5 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260807 19:48:46.114862 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be89699a1aff43f1a97b936b3154c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.092317 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ffb51072a064989a2056cfa1484f4a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.116781 8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be89699a1aff43f1a97b936b3154c2d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.125984 8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301b269c068f46ebae7e1c1d55e501f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.125921 8740 leader_election.cc:343] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.127054 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:46.127835 9107 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:46.131661 9106 tablet_bootstrap.cc:492] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:46.143699 9099 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:46.145200 8894 tablet_service.cc:1511] Processing CreateTablet for tablet 17510c7d48eb44ceb3d8c645ceaa28fc (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260807 19:48:46.145130 8773 tablet_service.cc:1511] Processing CreateTablet for tablet 17510c7d48eb44ceb3d8c645ceaa28fc (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260807 19:48:46.138970 9101 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:46.135640 8997 tablet_service.cc:1511] Processing CreateTablet for tablet 301b269c068f46ebae7e1c1d55e501f1 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260807 19:48:46.146785 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17510c7d48eb44ceb3d8c645ceaa28fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.146495 9101 ts_tablet_manager.cc:1403] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.103s user 0.009s sys 0.004s
I20260807 19:48:46.147269 8997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 301b269c068f46ebae7e1c1d55e501f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.143199 9117 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.149107 8894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17510c7d48eb44ceb3d8c645ceaa28fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.150054 9117 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.150980 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:46.151624 8915 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.153096 9101 raft_consensus.cc:359] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.158509 9106 ts_tablet_manager.cc:1403] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.146s user 0.011s sys 0.003s
I20260807 19:48:46.163877 8888 tablet_service.cc:1511] Processing CreateTablet for tablet d0e0dcc92bf04b66a4b58d4e68b01703 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260807 19:48:46.164757 9003 tablet_service.cc:1511] Processing CreateTablet for tablet 17510c7d48eb44ceb3d8c645ceaa28fc (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260807 19:48:46.165411 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e0dcc92bf04b66a4b58d4e68b01703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.166214 9106 raft_consensus.cc:359] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.167866 9106 raft_consensus.cc:385] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.155221 9099 tablet_bootstrap.cc:654] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.162604 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:46.167306 9101 raft_consensus.cc:385] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.168984 9101 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.168277 9106 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
W20260807 19:48:46.172281 8740 leader_election.cc:343] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.174961 9106 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.176203 8992 tablet_service.cc:1511] Processing CreateTablet for tablet d0e0dcc92bf04b66a4b58d4e68b01703 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260807 19:48:46.178037 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e0dcc92bf04b66a4b58d4e68b01703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.171594 8766 tablet_service.cc:1511] Processing CreateTablet for tablet d0e0dcc92bf04b66a4b58d4e68b01703 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260807 19:48:46.181149 9003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17510c7d48eb44ceb3d8c645ceaa28fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.181867 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0e0dcc92bf04b66a4b58d4e68b01703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.159659 8915 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:46.176498 9101 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.188772 9109 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.189271 9109 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.190200 9101 ts_tablet_manager.cc:1434] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.043s user 0.003s sys 0.003s
I20260807 19:48:46.191291 9101 tablet_bootstrap.cc:492] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:46.194698 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:46.195562 9117 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:46.196025 9117 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:46.196868 9117 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.201436 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.201468 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:48:46.203442 8856 leader_election.cc:343] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:46.204250 8849 leader_election.cc:343] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.204731 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:46.205818 9108 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:46.206506 9109 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.208925 8771 tablet_service.cc:1511] Processing CreateTablet for tablet 9d0dfd76f4dc484499bc12ddf5871c48 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260807 19:48:46.210485 8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d0dfd76f4dc484499bc12ddf5871c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.205958 9106 ts_tablet_manager.cc:1434] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.043s user 0.007s sys 0.000s
I20260807 19:48:46.218310 8630 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:46.224444 8900 tablet_service.cc:1511] Processing CreateTablet for tablet 9d0dfd76f4dc484499bc12ddf5871c48 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260807 19:48:46.227283 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d0dfd76f4dc484499bc12ddf5871c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.217903 9101 tablet_bootstrap.cc:654] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.239224 9099 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:46.239977 9099 ts_tablet_manager.cc:1403] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.115s user 0.011s sys 0.000s
I20260807 19:48:46.242791 9099 raft_consensus.cc:359] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.266295 9099 raft_consensus.cc:385] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.266433 9101 tablet_bootstrap.cc:492] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:46.266705 9099 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.267014 9101 ts_tablet_manager.cc:1403] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.076s user 0.013s sys 0.000s
I20260807 19:48:46.267550 9099 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.252527 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 9d0dfd76f4dc484499bc12ddf5871c48 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260807 19:48:46.269387 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d0dfd76f4dc484499bc12ddf5871c48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.269244 8881 tablet_service.cc:1511] Processing CreateTablet for tablet 7abc25df9cc247e8b0354c9758c11208 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260807 19:48:46.269927 9002 tablet_service.cc:1511] Processing CreateTablet for tablet 90fa71b3b19f407d8ede2571544c265c (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260807 19:48:46.246582 9106 tablet_bootstrap.cc:492] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:46.256755 8765 tablet_service.cc:1511] Processing CreateTablet for tablet 90fa71b3b19f407d8ede2571544c265c (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260807 19:48:46.275488 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90fa71b3b19f407d8ede2571544c265c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.276847 8771 tablet_service.cc:1511] Processing CreateTablet for tablet be133eef91634be592094b95357ee200 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260807 19:48:46.278450 8770 tablet_service.cc:1511] Processing CreateTablet for tablet 7abc25df9cc247e8b0354c9758c11208 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260807 19:48:46.293306 8773 tablet_service.cc:1511] Processing CreateTablet for tablet fbfe42e0961647279ca7973da14c62a0 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260807 19:48:46.271728 8900 tablet_service.cc:1511] Processing CreateTablet for tablet 90fa71b3b19f407d8ede2571544c265c (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260807 19:48:46.272992 8997 tablet_service.cc:1511] Processing CreateTablet for tablet 7abc25df9cc247e8b0354c9758c11208 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260807 19:48:46.270059 9101 raft_consensus.cc:359] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.295248 9101 raft_consensus.cc:385] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.295571 9101 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.295961 8992 tablet_service.cc:1511] Processing CreateTablet for tablet fbfe42e0961647279ca7973da14c62a0 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260807 19:48:46.296415 9101 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.298017 8764 tablet_service.cc:1511] Processing CreateTablet for tablet 479d60d809ec458397b1360e820ed150 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260807 19:48:46.298732 9101 ts_tablet_manager.cc:1434] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260807 19:48:46.276391 9002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90fa71b3b19f407d8ede2571544c265c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.279929 8771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be133eef91634be592094b95357ee200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.300635 8764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 479d60d809ec458397b1360e820ed150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.280730 9117 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.301033 8885 tablet_service.cc:1511] Processing CreateTablet for tablet fbfe42e0961647279ca7973da14c62a0 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260807 19:48:46.301200 9117 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.270394 9099 ts_tablet_manager.cc:1434] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260807 19:48:46.303179 9001 tablet_service.cc:1511] Processing CreateTablet for tablet 479d60d809ec458397b1360e820ed150 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260807 19:48:46.304291 9099 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:46.304567 8767 tablet_service.cc:1511] Processing CreateTablet for tablet 0ba47e79caa54c409c53304a66ce0160 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260807 19:48:46.300042 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfe42e0961647279ca7973da14c62a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.306192 8767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba47e79caa54c409c53304a66ce0160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.306566 8997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abc25df9cc247e8b0354c9758c11208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.270553 8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abc25df9cc247e8b0354c9758c11208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.281260 9106 tablet_bootstrap.cc:654] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.280910 8894 tablet_service.cc:1511] Processing CreateTablet for tablet be133eef91634be592094b95357ee200 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260807 19:48:46.281978 9116 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.282418 9107 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.283231 9003 tablet_service.cc:1511] Processing CreateTablet for tablet be133eef91634be592094b95357ee200 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260807 19:48:46.285377 8766 tablet_service.cc:1511] Processing CreateTablet for tablet b89518940e0d4f7c848bc64e87de7e25 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260807 19:48:46.289124 8884 tablet_service.cc:1511] Processing CreateTablet for tablet b89518940e0d4f7c848bc64e87de7e25 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260807 19:48:46.300114 8888 tablet_service.cc:1511] Processing CreateTablet for tablet 479d60d809ec458397b1360e820ed150 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260807 19:48:46.290068 9006 tablet_service.cc:1511] Processing CreateTablet for tablet b89518940e0d4f7c848bc64e87de7e25 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260807 19:48:46.310081 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfe42e0961647279ca7973da14c62a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.316394 9109 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.316257 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.316900 9109 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:46.317945 8740 leader_election.cc:343] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.318993 9109 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.319676 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.311069 8898 tablet_service.cc:1511] Processing CreateTablet for tablet 0ba47e79caa54c409c53304a66ce0160 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260807 19:48:46.309290 9107 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.322062 8900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90fa71b3b19f407d8ede2571544c265c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.309075 9116 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.323218 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:46.324455 8894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be133eef91634be592094b95357ee200. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.324932 8740 leader_election.cc:343] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.325371 8740 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:46.325783 8898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba47e79caa54c409c53304a66ce0160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.323200 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.330057 9006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89518940e0d4f7c848bc64e87de7e25. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.332129 8856 leader_election.cc:343] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.332665 9107 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.332940 9001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 479d60d809ec458397b1360e820ed150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.333323 9101 tablet_bootstrap.cc:492] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:46.332683 9099 tablet_bootstrap.cc:654] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.336597 9116 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:46.337357 8991 tablet_service.cc:1511] Processing CreateTablet for tablet 0ba47e79caa54c409c53304a66ce0160 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260807 19:48:46.303236 9117 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.341609 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89518940e0d4f7c848bc64e87de7e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.344269 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7abc25df9cc247e8b0354c9758c11208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.344534 8769 tablet_service.cc:1511] Processing CreateTablet for tablet 26e0472ef12247a4a432a786e2656de3 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260807 19:48:46.344906 8899 tablet_service.cc:1511] Processing CreateTablet for tablet 26e0472ef12247a4a432a786e2656de3 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260807 19:48:46.345883 8769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e0472ef12247a4a432a786e2656de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.341478 8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b89518940e0d4f7c848bc64e87de7e25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.347375 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 479d60d809ec458397b1360e820ed150. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.348953 8899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e0472ef12247a4a432a786e2656de3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.339538 8849 leader_election.cc:343] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.351275 8849 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:46.339836 9107 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:46.362829 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfe42e0961647279ca7973da14c62a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.364813 8996 tablet_service.cc:1511] Processing CreateTablet for tablet 26e0472ef12247a4a432a786e2656de3 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260807 19:48:46.367548 8774 tablet_service.cc:1511] Processing CreateTablet for tablet 80eb998e349c4e2885504025672242a7 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260807 19:48:46.369071 8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb998e349c4e2885504025672242a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.372619 9003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be133eef91634be592094b95357ee200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.378897 8889 tablet_service.cc:1511] Processing CreateTablet for tablet 80eb998e349c4e2885504025672242a7 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260807 19:48:46.381004 8994 tablet_service.cc:1511] Processing CreateTablet for tablet 80eb998e349c4e2885504025672242a7 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260807 19:48:46.382372 8994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb998e349c4e2885504025672242a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.380892 8889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80eb998e349c4e2885504025672242a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.395778 8768 tablet_service.cc:1511] Processing CreateTablet for tablet 300f918f185249e6895ab57edd060d31 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260807 19:48:46.399549 9101 tablet_bootstrap.cc:654] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.406623 8893 tablet_service.cc:1511] Processing CreateTablet for tablet 300f918f185249e6895ab57edd060d31 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260807 19:48:46.408149 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300f918f185249e6895ab57edd060d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.361006 9109 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:46.409137 8768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300f918f185249e6895ab57edd060d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.419514 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 300f918f185249e6895ab57edd060d31 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260807 19:48:46.420894 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300f918f185249e6895ab57edd060d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.432861 8996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e0472ef12247a4a432a786e2656de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.447247 8766 tablet_service.cc:1511] Processing CreateTablet for tablet ab8e617d098d425cbddc5d78da43ab99 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260807 19:48:46.450418 8991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba47e79caa54c409c53304a66ce0160. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.430451 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 105542us
I20260807 19:48:46.458344 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:46.460302 8740 leader_election.cc:343] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.466766 9116 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.467336 9116 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.469462 9116 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:46.469806 8765 tablet_service.cc:1511] Processing CreateTablet for tablet 8977afab19724738a872c7880e363557 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260807 19:48:46.471200 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:46.472824 8740 leader_election.cc:343] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.473291 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:46.464094 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e617d098d425cbddc5d78da43ab99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.480660 9137 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:46.473457 8992 tablet_service.cc:1511] Processing CreateTablet for tablet ab8e617d098d425cbddc5d78da43ab99 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260807 19:48:46.499758 8893 tablet_service.cc:1511] Processing CreateTablet for tablet 8977afab19724738a872c7880e363557 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260807 19:48:46.493345 8884 tablet_service.cc:1511] Processing CreateTablet for tablet ab8e617d098d425cbddc5d78da43ab99 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260807 19:48:46.501017 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e617d098d425cbddc5d78da43ab99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.505317 8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e617d098d425cbddc5d78da43ab99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.507165 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.508644 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.509300 8915 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
W20260807 19:48:46.510972 8972 leader_election.cc:343] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.511723 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:46.512682 9116 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:46.513041 9116 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:46.513368 9116 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.516985 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8977afab19724738a872c7880e363557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.521528 9106 tablet_bootstrap.cc:492] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:46.522085 9106 ts_tablet_manager.cc:1403] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.276s user 0.021s sys 0.007s
I20260807 19:48:46.524988 9099 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:46.525501 9099 ts_tablet_manager.cc:1403] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.221s user 0.024s sys 0.004s
I20260807 19:48:46.526088 9106 raft_consensus.cc:359] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.527060 9106 raft_consensus.cc:385] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.527362 9106 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.528096 9106 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.530704 9099 raft_consensus.cc:359] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.532610 9099 raft_consensus.cc:385] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.532929 9099 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.533689 9099 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.539903 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:46.541906 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:48:46.543776 8968 leader_election.cc:343] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.552449 8994 tablet_service.cc:1511] Processing CreateTablet for tablet 8977afab19724738a872c7880e363557 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260807 19:48:46.548676 8968 leader_election.cc:343] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.553961 8994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8977afab19724738a872c7880e363557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.553731 8968 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:46.554502 9099 ts_tablet_manager.cc:1434] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.029s user 0.007s sys 0.000s
I20260807 19:48:46.555723 9099 tablet_bootstrap.cc:492] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:46.561412 8774 tablet_service.cc:1511] Processing CreateTablet for tablet f4ace421d3564015bfd7eb6cff4b78fc (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260807 19:48:46.564616 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8977afab19724738a872c7880e363557. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.565356 8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ace421d3564015bfd7eb6cff4b78fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.569870 8893 tablet_service.cc:1511] Processing CreateTablet for tablet f4ace421d3564015bfd7eb6cff4b78fc (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260807 19:48:46.571513 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ace421d3564015bfd7eb6cff4b78fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.529565 9116 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.574767 9116 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:46.575577 9116 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:46.576642 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:46.577250 8915 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.586625 8773 tablet_service.cc:1511] Processing CreateTablet for tablet f02f58b5ca51497592868892f144492a (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260807 19:48:46.586980 9005 tablet_service.cc:1511] Processing CreateTablet for tablet f4ace421d3564015bfd7eb6cff4b78fc (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260807 19:48:46.590929 8888 tablet_service.cc:1511] Processing CreateTablet for tablet f02f58b5ca51497592868892f144492a (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260807 19:48:46.592267 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4ace421d3564015bfd7eb6cff4b78fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.592459 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02f58b5ca51497592868892f144492a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.594857 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46209.6us
I20260807 19:48:46.596571 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:48:46.598116 8968 leader_election.cc:343] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.598061 8997 tablet_service.cc:1511] Processing CreateTablet for tablet f02f58b5ca51497592868892f144492a (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260807 19:48:46.599594 8997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02f58b5ca51497592868892f144492a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.588184 8773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f02f58b5ca51497592868892f144492a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.610363 8766 tablet_service.cc:1511] Processing CreateTablet for tablet bf699a9379e64939a7048af91aef93ec (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260807 19:48:46.611776 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf699a9379e64939a7048af91aef93ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.621097 8894 tablet_service.cc:1511] Processing CreateTablet for tablet bf699a9379e64939a7048af91aef93ec (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260807 19:48:46.659479 9099 tablet_bootstrap.cc:654] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:46.673473 8740 consensus_peers.cc:597] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Couldn't send request to peer a38f4d6088cf4c8cb510200ca0555801. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:46.674535 8894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf699a9379e64939a7048af91aef93ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.675604 9137 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:46.680037 8991 tablet_service.cc:1511] Processing CreateTablet for tablet bf699a9379e64939a7048af91aef93ec (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260807 19:48:46.682655 8991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf699a9379e64939a7048af91aef93ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.684217 9101 tablet_bootstrap.cc:492] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:46.684899 9101 ts_tablet_manager.cc:1403] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.352s user 0.017s sys 0.013s
I20260807 19:48:46.686841 8915 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:46.687112 9106 ts_tablet_manager.cc:1434] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.165s user 0.026s sys 0.057s
I20260807 19:48:46.688328 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:46.689723 9145 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:46.690249 9145 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:46.690452 8893 tablet_service.cc:1511] Processing CreateTablet for tablet 7f72bb6e69514b33881da72833a29378 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260807 19:48:46.691732 9106 tablet_bootstrap.cc:492] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:46.691496 8770 tablet_service.cc:1511] Processing CreateTablet for tablet 7f72bb6e69514b33881da72833a29378 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260807 19:48:46.692962 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f72bb6e69514b33881da72833a29378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.691916 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f72bb6e69514b33881da72833a29378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.691349 9145 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.698746 9106 tablet_bootstrap.cc:654] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.700616 9101 raft_consensus.cc:359] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.716045 9101 raft_consensus.cc:385] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.715826 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 7f72bb6e69514b33881da72833a29378 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260807 19:48:46.716387 9101 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.717181 9101 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.717222 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f72bb6e69514b33881da72833a29378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.719431 9101 ts_tablet_manager.cc:1434] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20260807 19:48:46.726013 8769 tablet_service.cc:1511] Processing CreateTablet for tablet bff76e2c6d6f4e35bf33640e6f4312a0 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260807 19:48:46.727447 8769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff76e2c6d6f4e35bf33640e6f4312a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.736132 9101 tablet_bootstrap.cc:492] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:46.750955 8768 tablet_service.cc:1511] Processing CreateTablet for tablet ed0dd3f602c145219dc8717e182ff568 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260807 19:48:46.752311 8768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0dd3f602c145219dc8717e182ff568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.752741 8885 tablet_service.cc:1511] Processing CreateTablet for tablet ed0dd3f602c145219dc8717e182ff568 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260807 19:48:46.756421 8992 tablet_service.cc:1511] Processing CreateTablet for tablet ed0dd3f602c145219dc8717e182ff568 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260807 19:48:46.757812 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0dd3f602c145219dc8717e182ff568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.756807 8893 tablet_service.cc:1511] Processing CreateTablet for tablet bff76e2c6d6f4e35bf33640e6f4312a0 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260807 19:48:46.762235 9005 tablet_service.cc:1511] Processing CreateTablet for tablet bff76e2c6d6f4e35bf33640e6f4312a0 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260807 19:48:46.763548 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff76e2c6d6f4e35bf33640e6f4312a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.775723 8889 tablet_service.cc:1511] Processing CreateTablet for tablet 915c924a58ad4f3cbf22affe06485a15 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260807 19:48:46.777192 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0dd3f602c145219dc8717e182ff568. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.777546 8774 tablet_service.cc:1511] Processing CreateTablet for tablet 915c924a58ad4f3cbf22affe06485a15 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260807 19:48:46.779160 8774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915c924a58ad4f3cbf22affe06485a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.784425 8996 tablet_service.cc:1511] Processing CreateTablet for tablet 915c924a58ad4f3cbf22affe06485a15 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260807 19:48:46.785977 8996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915c924a58ad4f3cbf22affe06485a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.788806 9109 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.789388 9109 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.791582 9109 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.794153 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:46.794855 8789 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:46.811997 9106 tablet_bootstrap.cc:492] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:46.812704 9106 ts_tablet_manager.cc:1403] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.121s user 0.015s sys 0.009s
I20260807 19:48:46.815735 9101 tablet_bootstrap.cc:654] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:46.803663 9099 tablet_bootstrap.cc:492] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:46.796274 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:46.808825 8889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 915c924a58ad4f3cbf22affe06485a15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.825225 9109 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:46.825685 9109 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:46.826047 8767 tablet_service.cc:1511] Processing CreateTablet for tablet fb33dc9f5a49409bb8817073ec1143da (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260807 19:48:46.827490 8767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb33dc9f5a49409bb8817073ec1143da. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:48:46.837031 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:46.839011 8888 tablet_service.cc:1511] Processing CreateTablet for tablet fb33dc9f5a49409bb8817073ec1143da (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260807 19:48:46.840090 8632 catalog_manager.cc:5697] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:46.844408 9099 ts_tablet_manager.cc:1403] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.289s user 0.015s sys 0.029s
I20260807 19:48:46.848810 9099 raft_consensus.cc:359] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.849624 9099 raft_consensus.cc:385] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.849933 9099 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.850432 8884 tablet_service.cc:1511] Processing CreateTablet for tablet fe08dfdbfbdb4fd08342efd539590bf2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260807 19:48:46.826005 9109 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.864730 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.816093 9106 raft_consensus.cc:359] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.873481 8769 tablet_service.cc:1511] Processing CreateTablet for tablet 8666b7cb7a05408a8223317d32823bc9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260807 19:48:46.873733 8991 tablet_service.cc:1511] Processing CreateTablet for tablet fb33dc9f5a49409bb8817073ec1143da (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260807 19:48:46.874917 8769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8666b7cb7a05408a8223317d32823bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.875231 8991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb33dc9f5a49409bb8817073ec1143da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.871840 8884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe08dfdbfbdb4fd08342efd539590bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.877686 8994 tablet_service.cc:1511] Processing CreateTablet for tablet fe08dfdbfbdb4fd08342efd539590bf2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260807 19:48:46.878816 8894 tablet_service.cc:1511] Processing CreateTablet for tablet 8666b7cb7a05408a8223317d32823bc9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260807 19:48:46.879349 8994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe08dfdbfbdb4fd08342efd539590bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.870054 9106 raft_consensus.cc:385] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:46.879739 9106 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:46.880648 9106 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:46.881559 8856 leader_election.cc:343] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.881703 8767 tablet_service.cc:1511] Processing CreateTablet for tablet fe08dfdbfbdb4fd08342efd539590bf2 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260807 19:48:46.883028 8767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe08dfdbfbdb4fd08342efd539590bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.886336 8894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8666b7cb7a05408a8223317d32823bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.894160 9106 ts_tablet_manager.cc:1434] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.081s user 0.007s sys 0.000s
I20260807 19:48:46.895364 9106 tablet_bootstrap.cc:492] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:46.905831 8991 tablet_service.cc:1511] Processing CreateTablet for tablet 8666b7cb7a05408a8223317d32823bc9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260807 19:48:46.906661 8898 tablet_service.cc:1511] Processing CreateTablet for tablet 1c01405ca90e4d1ebb91f47316c28aa9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260807 19:48:46.907310 8991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8666b7cb7a05408a8223317d32823bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.909027 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 1c01405ca90e4d1ebb91f47316c28aa9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260807 19:48:46.909384 8766 tablet_service.cc:1511] Processing CreateTablet for tablet 1c01405ca90e4d1ebb91f47316c28aa9 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260807 19:48:46.910223 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c01405ca90e4d1ebb91f47316c28aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.910554 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c01405ca90e4d1ebb91f47316c28aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.912168 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb33dc9f5a49409bb8817073ec1143da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.912473 9145 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.912886 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bff76e2c6d6f4e35bf33640e6f4312a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.912961 9145 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.915096 8898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c01405ca90e4d1ebb91f47316c28aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.915465 9145 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:46.916800 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.917214 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:46.917843 8789 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
W20260807 19:48:46.918506 8972 leader_election.cc:343] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.919373 8968 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:46.870985 9099 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.921485 9145 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:46.922003 9145 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:46.922399 9145 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.923467 9099 ts_tablet_manager.cc:1434] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.079s user 0.004s sys 0.004s
I20260807 19:48:46.934477 9109 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.938107 8765 tablet_service.cc:1511] Processing CreateTablet for tablet 95f757dd3f63420fad091c3802390158 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260807 19:48:46.940567 8881 tablet_service.cc:1511] Processing CreateTablet for tablet 95f757dd3f63420fad091c3802390158 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260807 19:48:46.941548 8765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f757dd3f63420fad091c3802390158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.941910 8881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f757dd3f63420fad091c3802390158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.943298 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:46.944316 8789 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.947818 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:46.948520 9109 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:48:46.950130 8856 leader_election.cc:343] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.956058 9099 tablet_bootstrap.cc:492] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:46.969228 8994 tablet_service.cc:1511] Processing CreateTablet for tablet 95f757dd3f63420fad091c3802390158 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260807 19:48:46.971453 8994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95f757dd3f63420fad091c3802390158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:46.973968 9137 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.952991 8789 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:46.976420 9137 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.959995 9145 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.983004 9107 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:46.983515 9107 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:46.984130 9137 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.985002 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:46.985594 8790 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.987993 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.988669 8913 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:46.989269 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:46.989681 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.990895 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:46.993721 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:46.994592 9137 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 19:48:46.994692 8740 leader_election.cc:343] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:46.995003 9137 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:46.995385 9137 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:46.995728 8790 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:46.985538 9107 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:46.998773 9101 tablet_bootstrap.cc:492] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:46.999293 9101 ts_tablet_manager.cc:1403] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.279s user 0.019s sys 0.002s
W20260807 19:48:46.999728 8740 leader_election.cc:343] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.000612 9146 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.000857 8885 tablet_service.cc:1511] Processing CreateTablet for tablet 138f0e8a31994c549dd743c9ba49d6ee (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260807 19:48:47.001143 9146 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.002272 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138f0e8a31994c549dd743c9ba49d6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.003350 9146 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.005615 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:46.999317 9099 tablet_bootstrap.cc:654] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:47.007331 8968 leader_election.cc:343] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:47.000759 8740 leader_election.cc:343] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.008462 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.009315 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.009656 9107 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:48:47.011103 8972 leader_election.cc:343] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.011615 8972 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:47.012521 9146 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.020215 8992 tablet_service.cc:1511] Processing CreateTablet for tablet 138f0e8a31994c549dd743c9ba49d6ee (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260807 19:48:47.021953 8992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138f0e8a31994c549dd743c9ba49d6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.031749 8766 tablet_service.cc:1511] Processing CreateTablet for tablet 138f0e8a31994c549dd743c9ba49d6ee (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260807 19:48:47.033355 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138f0e8a31994c549dd743c9ba49d6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.044697 8968 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:47.046736 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.047957 9109 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.048486 9109 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:47.048713 8770 tablet_service.cc:1511] Processing CreateTablet for tablet afb6953f02d1486aaee40c8901954290 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260807 19:48:47.049342 9109 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.050148 8770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb6953f02d1486aaee40c8901954290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.052556 9146 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.053099 9146 raft_consensus.cc:697] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:47.044569 9145 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.054896 9146 consensus_queue.cc:237] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.068267 8899 tablet_service.cc:1511] Processing CreateTablet for tablet afb6953f02d1486aaee40c8901954290 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260807 19:48:47.069743 8899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb6953f02d1486aaee40c8901954290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.068912 8994 tablet_service.cc:1511] Processing CreateTablet for tablet afb6953f02d1486aaee40c8901954290 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260807 19:48:47.071461 8994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb6953f02d1486aaee40c8901954290. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.077097 8766 tablet_service.cc:1511] Processing CreateTablet for tablet 49088520f262493898ad206e1e077f2b (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260807 19:48:47.079293 8968 consensus_peers.cc:597] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 -> Peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Couldn't send request to peer 8e9ad2b5e2a94d4faff90af735c6d792. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:47.080422 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49088520f262493898ad206e1e077f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.080683 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:47.087419 9137 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.061764 9106 tablet_bootstrap.cc:700] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent opening tablet: real 0.102s user 0.006s sys 0.004s
I20260807 19:48:47.079339 9101 raft_consensus.cc:359] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:47.093740 8972 leader_election.cc:343] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.100304 8885 tablet_service.cc:1511] Processing CreateTablet for tablet 49088520f262493898ad206e1e077f2b (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260807 19:48:47.101387 8885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49088520f262493898ad206e1e077f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.099397 9101 raft_consensus.cc:385] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.102273 9101 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.107692 9101 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.108860 8766 tablet_service.cc:1511] Processing CreateTablet for tablet ef1e484df4724d9896a8f46b89923261 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260807 19:48:47.115232 9099 tablet_bootstrap.cc:492] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.115409 8766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef1e484df4724d9896a8f46b89923261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.115839 9099 ts_tablet_manager.cc:1403] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.160s user 0.012s sys 0.012s
I20260807 19:48:47.101783 9106 tablet_bootstrap.cc:654] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.117951 8630 catalog_manager.cc:5697] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.119082 9099 raft_consensus.cc:359] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.119827 9099 raft_consensus.cc:385] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.120126 9099 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.121117 9099 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.126006 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:47.126670 8915 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.126775 8769 tablet_service.cc:1511] Processing CreateTablet for tablet 10901e14a5e8471291ad1d7c203b6740 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260807 19:48:47.127537 9005 tablet_service.cc:1511] Processing CreateTablet for tablet 10901e14a5e8471291ad1d7c203b6740 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260807 19:48:47.128260 8769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10901e14a5e8471291ad1d7c203b6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.128686 9005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10901e14a5e8471291ad1d7c203b6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.136858 8915 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:47.138309 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:47.139319 9107 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.139858 9107 raft_consensus.cc:697] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:47.140383 9137 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.111472 9101 ts_tablet_manager.cc:1434] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.110s user 0.012s sys 0.000s
I20260807 19:48:47.113322 8991 tablet_service.cc:1511] Processing CreateTablet for tablet ef1e484df4724d9896a8f46b89923261 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260807 19:48:47.141010 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:47.141877 9101 tablet_bootstrap.cc:492] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.141999 8991 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef1e484df4724d9896a8f46b89923261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.141010 9107 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:47.142817 8740 leader_election.cc:343] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.113322 8888 tablet_service.cc:1511] Processing CreateTablet for tablet ef1e484df4724d9896a8f46b89923261 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260807 19:48:47.144800 8888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef1e484df4724d9896a8f46b89923261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.153194 9101 tablet_bootstrap.cc:654] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.154110 9109 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.168751 9109 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.169948 8893 tablet_service.cc:1511] Processing CreateTablet for tablet 10901e14a5e8471291ad1d7c203b6740 (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260807 19:48:47.171778 8893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10901e14a5e8471291ad1d7c203b6740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.171864 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.173647 9099 ts_tablet_manager.cc:1434] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
W20260807 19:48:47.173583 8849 leader_election.cc:343] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.174638 9109 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.175905 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.176611 9023 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:47.178695 8856 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:47.179905 9109 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.174738 8996 tablet_service.cc:1511] Processing CreateTablet for tablet 49088520f262493898ad206e1e077f2b (DEFAULT_TABLE table=test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260807 19:48:47.186137 9099 tablet_bootstrap.cc:492] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.183089 9109 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.190491 9109 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.192484 8996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49088520f262493898ad206e1e077f2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:47.200002 8632 catalog_manager.cc:5697] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:47.202324 9099 tablet_bootstrap.cc:654] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.210234 9106 tablet_bootstrap.cc:492] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.210914 9106 ts_tablet_manager.cc:1403] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.316s user 0.024s sys 0.032s
I20260807 19:48:47.212241 9137 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.212797 9137 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.214524 9106 raft_consensus.cc:359] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.215433 9106 raft_consensus.cc:385] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.215811 9106 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.214975 9137 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.218972 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.219620 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.221513 8630 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.216704 9106 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:47.225416 8740 leader_election.cc:343] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:47.226811 8740 leader_election.cc:343] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.227231 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.227696 9106 ts_tablet_manager.cc:1434] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.016s user 0.003s sys 0.003s
I20260807 19:48:47.229000 9106 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.229900 9137 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.232527 9109 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.235895 9106 tablet_bootstrap.cc:654] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.246924 9099 tablet_bootstrap.cc:492] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.247426 9099 ts_tablet_manager.cc:1403] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.062s user 0.011s sys 0.006s
I20260807 19:48:47.247443 9101 tablet_bootstrap.cc:492] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.249944 9101 ts_tablet_manager.cc:1403] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.108s user 0.013s sys 0.005s
I20260807 19:48:47.250095 9099 raft_consensus.cc:359] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.250854 9099 raft_consensus.cc:385] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.251132 9099 raft_consensus.cc:740] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.251802 9099 consensus_queue.cc:260] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.254251 9099 ts_tablet_manager.cc:1434] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:48:47.256067 9099 tablet_bootstrap.cc:492] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.258468 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:48:47.260114 8849 leader_election.cc:343] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.261324 9101 raft_consensus.cc:359] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.261897 9101 raft_consensus.cc:385] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.262141 9101 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.262802 9101 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.263648 9099 tablet_bootstrap.cc:654] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.265403 9109 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.266705 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:47.267283 9023 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.270902 9106 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.271333 9106 ts_tablet_manager.cc:1403] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.005s
I20260807 19:48:47.273862 9106 raft_consensus.cc:359] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.274427 9106 raft_consensus.cc:385] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.274624 9106 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.275228 9106 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.276846 9101 ts_tablet_manager.cc:1434] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.027s user 0.005s sys 0.002s
I20260807 19:48:47.277282 9099 tablet_bootstrap.cc:492] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.277760 9099 ts_tablet_manager.cc:1403] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.003s
I20260807 19:48:47.277925 9101 tablet_bootstrap.cc:492] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.280499 9099 raft_consensus.cc:359] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.281244 9099 raft_consensus.cc:385] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.281546 9099 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.281749 9023 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:47.282208 9099 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.286813 8856 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:47.287361 9099 ts_tablet_manager.cc:1434] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:48:47.284452 9106 ts_tablet_manager.cc:1434] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.013s user 0.007s sys 0.001s
I20260807 19:48:47.287649 9109 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.288159 9109 raft_consensus.cc:697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:47.288578 9106 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.289057 9146 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:47.289048 9109 consensus_queue.cc:237] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.294493 9101 tablet_bootstrap.cc:654] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.297443 9099 tablet_bootstrap.cc:492] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.299082 9137 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.299662 9106 tablet_bootstrap.cc:654] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.299605 9137 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.301925 9137 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.304008 9099 tablet_bootstrap.cc:654] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.304138 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:47.306180 8740 leader_election.cc:343] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.312777 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:47.316227 8740 leader_election.cc:343] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.316793 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.320590 9137 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.321643 8629 catalog_manager.cc:5697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.331197 9101 tablet_bootstrap.cc:492] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.331866 9101 ts_tablet_manager.cc:1403] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.054s user 0.016s sys 0.003s
I20260807 19:48:47.334707 9101 raft_consensus.cc:359] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.335428 9101 raft_consensus.cc:385] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.335717 9101 raft_consensus.cc:740] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.336170 9099 tablet_bootstrap.cc:492] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.336704 9099 ts_tablet_manager.cc:1403] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.039s user 0.020s sys 0.017s
I20260807 19:48:47.337105 9106 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.337648 9106 ts_tablet_manager.cc:1403] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.049s user 0.007s sys 0.014s
I20260807 19:48:47.339704 9099 raft_consensus.cc:359] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.340487 9099 raft_consensus.cc:385] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.340777 9099 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.341069 9106 raft_consensus.cc:359] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.341807 9106 raft_consensus.cc:385] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.341619 9099 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.347944 9099 ts_tablet_manager.cc:1434] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.011s user 0.002s sys 0.004s
I20260807 19:48:47.348536 9101 consensus_queue.cc:260] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.349192 9099 tablet_bootstrap.cc:492] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.350715 9101 ts_tablet_manager.cc:1434] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260807 19:48:47.351750 9101 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.342104 9106 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.355991 9099 tablet_bootstrap.cc:654] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.356753 9106 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.365435 9146 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.366019 9146 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.369051 9116 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.369683 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.369599 9116 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.370579 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.371555 8790 raft_consensus.cc:2393] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:47.371886 9116 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.368287 9146 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.373642 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.373883 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:47.375175 9116 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:47.375373 8972 leader_election.cc:343] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.376928 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.377774 8790 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:47.380227 8968 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:47.382185 9116 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.382572 9146 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.383095 9146 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.383456 9146 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.384907 9106 ts_tablet_manager.cc:1434] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.047s user 0.009s sys 0.000s
I20260807 19:48:47.392921 9146 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.396031 9106 tablet_bootstrap.cc:492] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.396670 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:47.397473 8790 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.399892 9101 tablet_bootstrap.cc:654] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.404791 9146 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.406126 9099 tablet_bootstrap.cc:492] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.406236 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:47.406456 8790 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:47.406711 9099 ts_tablet_manager.cc:1403] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.058s user 0.018s sys 0.034s
I20260807 19:48:47.407913 8968 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:48:47.408589 8972 leader_election.cc:343] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.409699 9145 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.410310 9145 raft_consensus.cc:697] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:47.410162 9099 raft_consensus.cc:359] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.410840 9106 tablet_bootstrap.cc:654] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.410882 9099 raft_consensus.cc:385] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.411476 9099 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.412225 9099 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.414543 9099 ts_tablet_manager.cc:1434] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:48:47.416476 9101 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.417137 9101 ts_tablet_manager.cc:1403] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.066s user 0.012s sys 0.000s
I20260807 19:48:47.411491 9145 consensus_queue.cc:237] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.418147 9099 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.422735 9101 raft_consensus.cc:359] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.423554 9101 raft_consensus.cc:385] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.423878 9101 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.424983 9099 tablet_bootstrap.cc:654] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.425448 9101 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.428960 9101 ts_tablet_manager.cc:1434] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260807 19:48:47.431816 9101 tablet_bootstrap.cc:492] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.438536 9101 tablet_bootstrap.cc:654] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.440625 8632 catalog_manager.cc:5697] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:47.446198 9101 tablet_bootstrap.cc:492] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.446704 9101 ts_tablet_manager.cc:1403] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260807 19:48:47.446455 9109 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:47.450784 9106 tablet_bootstrap.cc:492] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.451300 9106 ts_tablet_manager.cc:1403] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.056s user 0.018s sys 0.013s
I20260807 19:48:47.451712 9101 raft_consensus.cc:359] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.452569 9101 raft_consensus.cc:385] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.452917 9101 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.453769 9101 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.455848 9106 raft_consensus.cc:359] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.456203 9101 ts_tablet_manager.cc:1434] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:48:47.456420 9106 raft_consensus.cc:385] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.456727 9106 raft_consensus.cc:740] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.457394 9101 tablet_bootstrap.cc:492] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.451297 9099 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.458071 9099 ts_tablet_manager.cc:1403] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.006s
I20260807 19:48:47.460717 9099 raft_consensus.cc:359] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.461403 9099 raft_consensus.cc:385] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.461684 9099 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.462350 9099 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.463238 9101 tablet_bootstrap.cc:654] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.457515 9106 consensus_queue.cc:260] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.464196 9099 ts_tablet_manager.cc:1434] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260807 19:48:47.466909 9106 ts_tablet_manager.cc:1434] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20260807 19:48:47.470515 9106 tablet_bootstrap.cc:492] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.474236 9109 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:47.478886 9106 tablet_bootstrap.cc:654] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.483438 9099 tablet_bootstrap.cc:492] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.492317 9109 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.492998 9109 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.495440 9109 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.497983 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.499260 9101 tablet_bootstrap.cc:492] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.499818 9101 ts_tablet_manager.cc:1403] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.002s
I20260807 19:48:47.500226 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.501027 8789 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:47.502983 9106 tablet_bootstrap.cc:492] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
W20260807 19:48:47.503024 8856 leader_election.cc:343] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.503701 9106 ts_tablet_manager.cc:1403] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.007s
W20260807 19:48:47.503895 8972 consensus_peers.cc:597] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Couldn't send request to peer 671d8754170646eaac5748745eba58b4. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:47.505075 8849 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.506126 9109 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.506585 9109 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.506850 9106 raft_consensus.cc:359] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.507893 9106 raft_consensus.cc:385] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.506963 9109 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.508891 9106 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.509873 9106 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.512216 9106 ts_tablet_manager.cc:1434] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260807 19:48:47.513236 9106 tablet_bootstrap.cc:492] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.503155 9101 raft_consensus.cc:359] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.513880 9101 raft_consensus.cc:385] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.514189 9101 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.517453 9101 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.519385 9106 tablet_bootstrap.cc:654] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.519552 9101 ts_tablet_manager.cc:1434] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260807 19:48:47.519891 8789 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.538147 9099 tablet_bootstrap.cc:654] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.540515 9137 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.541169 9137 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.543578 9137 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.545866 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.546561 9023 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:47.547768 9106 tablet_bootstrap.cc:492] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.548334 9106 ts_tablet_manager.cc:1403] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.035s user 0.018s sys 0.014s
I20260807 19:48:47.548894 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.549587 8915 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:47.547915 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:47.551358 9106 raft_consensus.cc:359] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.552147 9106 raft_consensus.cc:385] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.552459 9106 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.552560 9137 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.552987 9137 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.553486 9137 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.552991 9101 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.553365 9106 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.521974 9109 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.562841 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:47.563481 8790 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.564553 9137 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.566350 9106 ts_tablet_manager.cc:1434] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260807 19:48:47.566710 9137 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.570310 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:47.571000 8915 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.575759 8790 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:47.576751 9192 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.576990 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:47.580991 9023 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.581012 9192 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.583752 9101 tablet_bootstrap.cc:654] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.585378 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.586062 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.586154 9022 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:47.586776 8788 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:47.588398 8856 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:47.589567 9156 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.589949 9156 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.590284 9156 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.595234 8915 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:47.577847 9106 tablet_bootstrap.cc:492] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.578686 9109 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.597251 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:47.598290 9107 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.598737 9107 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:47.599578 9107 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.579789 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:48:47.605221 8856 leader_election.cc:343] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.607666 9106 tablet_bootstrap.cc:654] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.608964 9156 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.619014 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:47.619773 9027 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.622413 9099 tablet_bootstrap.cc:492] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.623039 9099 ts_tablet_manager.cc:1403] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.140s user 0.036s sys 0.023s
I20260807 19:48:47.613056 9192 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.615139 8849 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.628091 9099 raft_consensus.cc:359] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.628963 9099 raft_consensus.cc:385] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.629266 9099 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.630134 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:47.630780 8789 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.632253 9192 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.633888 9192 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:47.634479 9027 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:47.600418 9023 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:47.636049 8856 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:47.635776 8630 catalog_manager.cc:5697] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.637297 9145 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.636230 9156 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.637792 9145 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.635177 9106 tablet_bootstrap.cc:492] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.634956 9192 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.638989 9106 ts_tablet_manager.cc:1403] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.061s user 0.017s sys 0.016s
I20260807 19:48:47.638985 9109 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.639647 9109 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:47.639943 9145 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.650370 9106 raft_consensus.cc:359] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.651129 9106 raft_consensus.cc:385] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.651423 9106 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.651789 9137 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:47.652103 9106 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.654241 9106 ts_tablet_manager.cc:1434] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.015s user 0.007s sys 0.002s
I20260807 19:48:47.655212 9106 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.644286 9181 consensus_queue.cc:1048] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260807 19:48:47.640451 9099 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.662750 9099 ts_tablet_manager.cc:1434] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.039s user 0.007s sys 0.005s
I20260807 19:48:47.663862 9099 tablet_bootstrap.cc:492] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.642932 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.669245 9109 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.671491 9106 tablet_bootstrap.cc:654] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.673044 8789 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:47.674536 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.670210 9099 tablet_bootstrap.cc:654] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.676259 8968 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:47.677238 9101 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.677534 9145 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.677805 9101 ts_tablet_manager.cc:1403] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.125s user 0.025s sys 0.023s
I20260807 19:48:47.681298 9101 raft_consensus.cc:359] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.683718 9101 raft_consensus.cc:385] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.684633 9101 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.686264 9099 tablet_bootstrap.cc:492] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.686784 9099 ts_tablet_manager.cc:1403] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.004s
I20260807 19:48:47.687970 9101 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.692444 9101 ts_tablet_manager.cc:1434] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260807 19:48:47.690135 9099 raft_consensus.cc:359] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.695472 9099 raft_consensus.cc:385] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.695875 9099 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.693915 9106 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.695339 9101 tablet_bootstrap.cc:492] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.696703 9106 ts_tablet_manager.cc:1403] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.042s user 0.017s sys 0.002s
I20260807 19:48:47.697005 9099 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.699410 9099 ts_tablet_manager.cc:1434] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260807 19:48:47.700632 9099 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.700045 9106 raft_consensus.cc:359] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.702232 9106 raft_consensus.cc:385] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.702556 9106 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.711452 9023 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.728858 9109 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.729164 9107 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:47.729521 9109 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.733397 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.734513 8790 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:47.736099 8849 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:47.737226 9156 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.737644 9156 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.737983 9156 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.748181 9099 tablet_bootstrap.cc:654] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.751555 9101 tablet_bootstrap.cc:654] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.752449 8630 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.753144 9109 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.753845 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.753993 9106 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.754582 9027 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:47.756531 9106 ts_tablet_manager.cc:1434] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.059s user 0.010s sys 0.008s
I20260807 19:48:47.756865 9156 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.757835 9106 tablet_bootstrap.cc:492] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.763141 9107 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.763769 9107 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.766907 9107 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.766777 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:47.768891 8790 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.771170 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.771900 9027 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:47.774510 9109 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:47.775971 9107 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.786660 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:47.771186 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.788537 9117 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.788961 9117 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.789309 9117 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:47.790226 8740 leader_election.cc:343] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.794176 9156 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.786437 9107 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.797755 9107 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.797652 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:47.798404 9023 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.799026 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.801653 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.803802 8740 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.805220 9107 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.806023 8790 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:47.810304 8630 catalog_manager.cc:5697] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.817180 8849 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:47.819195 9106 tablet_bootstrap.cc:654] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.824972 9101 tablet_bootstrap.cc:492] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:47.825621 9101 ts_tablet_manager.cc:1403] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.131s user 0.021s sys 0.014s
I20260807 19:48:47.824635 9117 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.820981 9156 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.830632 9156 raft_consensus.cc:697] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:47.826707 9145 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260807 19:48:47.833609 9023 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:47.835994 9101 raft_consensus.cc:359] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.836829 9101 raft_consensus.cc:385] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.837092 9101 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.837781 9101 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.833305 9156 consensus_queue.cc:237] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.840512 9101 ts_tablet_manager.cc:1434] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.015s user 0.005s sys 0.002s
I20260807 19:48:47.841488 9101 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:47.841409 9145 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.842126 9145 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.844069 9099 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:47.844586 9099 ts_tablet_manager.cc:1403] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.144s user 0.018s sys 0.036s
I20260807 19:48:47.854983 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:47.855154 9145 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.856462 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.857520 9117 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.858170 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.860073 8790 raft_consensus.cc:2393] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
W20260807 19:48:47.858645 8740 leader_election.cc:343] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:47.862270 9099 raft_consensus.cc:359] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.863965 9099 raft_consensus.cc:385] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.864216 9099 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.865020 9099 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.868727 8972 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:48:47.870483 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:47.870939 9146 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.872423 9099 ts_tablet_manager.cc:1434] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260807 19:48:47.873481 9099 tablet_bootstrap.cc:492] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:47.874581 8790 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.877164 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:47.877784 9027 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.877909 9106 tablet_bootstrap.cc:492] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:47.878427 9106 ts_tablet_manager.cc:1403] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.121s user 0.008s sys 0.005s
I20260807 19:48:47.879418 9200 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.879896 9200 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.882028 9101 tablet_bootstrap.cc:654] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.882028 9200 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.885083 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:47.887699 9146 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.896448 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.898566 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:47.899334 9027 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:47.899797 9200 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:47.900667 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:47.902149 9117 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:47.902650 9117 raft_consensus.cc:697] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:47.903556 9117 consensus_queue.cc:237] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.920760 9146 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.921439 9146 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.923635 9146 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:47.941334 9106 raft_consensus.cc:359] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.943238 9106 raft_consensus.cc:385] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:47.943555 9106 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:47.945001 9106 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.950695 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:47.952607 8789 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:47.954602 9099 tablet_bootstrap.cc:654] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:47.956791 9106 ts_tablet_manager.cc:1434] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.078s user 0.005s sys 0.006s
I20260807 19:48:47.957811 8968 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:47.958850 9146 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:47.959249 9146 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:47.959594 9146 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.961472 8630 catalog_manager.cc:5697] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:47.969143 9156 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:47.973779 9106 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:47.977975 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:47.978807 8915 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:47.985988 9156 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:47.991263 9156 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:47.993439 9156 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:47.994617 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.000066 9101 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.000794 9101 ts_tablet_manager.cc:1403] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.160s user 0.041s sys 0.036s
I20260807 19:48:48.003445 8629 catalog_manager.cc:5697] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:48.002012 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:48.007819 8915 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:47.995188 9146 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.009456 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.011061 9156 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.014200 9146 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:48.016006 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:48.016691 8913 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.028774 9146 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.018222 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:48.030241 8790 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.031695 9101 raft_consensus.cc:359] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.032404 9101 raft_consensus.cc:385] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.032702 9101 raft_consensus.cc:740] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.033512 9101 consensus_queue.cc:260] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.035459 9101 ts_tablet_manager.cc:1434] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.034s user 0.010s sys 0.005s
I20260807 19:48:48.032829 9099 tablet_bootstrap.cc:492] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.036079 9099 ts_tablet_manager.cc:1403] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.163s user 0.015s sys 0.015s
I20260807 19:48:48.036392 9101 tablet_bootstrap.cc:492] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.037029 9106 tablet_bootstrap.cc:654] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.039187 9099 raft_consensus.cc:359] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.041685 9099 raft_consensus.cc:385] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.041966 9099 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.042749 9099 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.043356 9101 tablet_bootstrap.cc:654] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.041111 8790 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:48.045149 9099 ts_tablet_manager.cc:1434] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260807 19:48:48.046051 8968 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:48.047199 9146 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:48.047703 9146 raft_consensus.cc:697] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:48.046126 9099 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.048628 9146 consensus_queue.cc:237] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.027910 8913 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:48.061923 9117 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.062548 9117 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.064810 9117 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.066442 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.066823 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.067160 8913 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
W20260807 19:48:48.068555 8740 leader_election.cc:343] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.069413 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.070976 8913 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.074208 9117 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:48.074635 9117 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:48.074965 9117 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.076956 8632 catalog_manager.cc:5697] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:48.093114 9220 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.093585 9099 tablet_bootstrap.cc:654] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.082373 9117 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.100545 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:48.102486 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:48:48.102882 8740 leader_election.cc:343] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.103740 9117 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.105415 8915 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:48.129346 9236 log.cc:927] Time spent T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260807 19:48:48.140954 8915 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:48.142043 9106 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.142555 9106 ts_tablet_manager.cc:1403] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.169s user 0.011s sys 0.010s
I20260807 19:48:48.142673 9099 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.143138 9099 ts_tablet_manager.cc:1403] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.097s user 0.021s sys 0.007s
I20260807 19:48:48.142570 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.144381 9117 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:48.144982 9117 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:48.146020 9117 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.151268 9099 raft_consensus.cc:359] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.152143 9099 raft_consensus.cc:385] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.152415 9220 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.152637 9099 raft_consensus.cc:740] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.153009 9220 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.153501 9099 consensus_queue.cc:260] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.155212 9220 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.156034 9099 ts_tablet_manager.cc:1434] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260807 19:48:48.156836 9156 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.157207 9099 tablet_bootstrap.cc:492] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.157330 9156 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.159565 9156 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.159449 9106 raft_consensus.cc:359] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.160140 9106 raft_consensus.cc:385] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.160442 9106 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.161773 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:48.165098 8856 leader_election.cc:343] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.165452 9099 tablet_bootstrap.cc:654] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.166925 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:48.167625 8790 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:48.167868 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.168844 9027 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:48.171324 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.170477 9106 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.181463 8632 catalog_manager.cc:5697] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:48.173095 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:48.185043 9027 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.185284 9220 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:48.185734 9220 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:48.186112 9220 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.188138 9099 tablet_bootstrap.cc:492] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.188699 9099 ts_tablet_manager.cc:1403] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.032s user 0.021s sys 0.008s
W20260807 19:48:48.189158 8740 leader_election.cc:343] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.178380 8849 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.191628 9106 ts_tablet_manager.cc:1434] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.035s user 0.002s sys 0.006s
I20260807 19:48:48.192616 9106 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.193276 9117 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:48.196313 9099 raft_consensus.cc:359] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.197044 9099 raft_consensus.cc:385] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.197319 9099 raft_consensus.cc:740] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.197968 9099 consensus_queue.cc:260] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.200033 9099 ts_tablet_manager.cc:1434] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260807 19:48:48.201288 9192 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:48.201699 9192 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:48.181502 9156 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260807 19:48:48.202149 9192 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.205511 9099 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.213271 9101 tablet_bootstrap.cc:492] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.213791 9101 ts_tablet_manager.cc:1403] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.178s user 0.025s sys 0.037s
I20260807 19:48:48.214370 9200 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.216920 9099 tablet_bootstrap.cc:654] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.227077 9192 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.231185 9220 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.236603 9117 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.239926 9117 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.242146 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:48.242780 9117 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.245971 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.247707 9156 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:48:48.248179 8740 leader_election.cc:343] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:48.249091 8740 leader_election.cc:343] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.250734 9200 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.252593 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.257742 9192 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.255055 9200 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260807 19:48:48.263234 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:48:48.263643 8740 leader_election.cc:343] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.263854 8789 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.264568 9220 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.274178 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:48.274906 9021 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.287016 9101 raft_consensus.cc:359] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.267283 8740 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.290077 9137 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.290251 9101 raft_consensus.cc:385] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.293005 9106 tablet_bootstrap.cc:654] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.293035 9101 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.294456 9101 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.296545 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:48.299990 9101 ts_tablet_manager.cc:1434] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.086s user 0.006s sys 0.001s
I20260807 19:48:48.301110 9101 tablet_bootstrap.cc:492] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
W20260807 19:48:48.303189 8856 leader_election.cc:343] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.312500 9021 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:48.335098 8789 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:48.342542 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:48.346741 9137 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:48.347456 9137 raft_consensus.cc:697] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:48.348593 9137 consensus_queue.cc:237] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.359714 9099 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.358337 9220 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260807 19:48:48.371064 9099 ts_tablet_manager.cc:1403] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.166s user 0.012s sys 0.018s
I20260807 19:48:48.373997 9099 raft_consensus.cc:359] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.363014 8915 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.356755 9101 tablet_bootstrap.cc:654] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.374864 9099 raft_consensus.cc:385] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.378310 9099 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.379272 9099 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.359448 8849 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.381753 9099 ts_tablet_manager.cc:1434] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260807 19:48:48.382682 9106 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.383251 9106 ts_tablet_manager.cc:1403] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.191s user 0.019s sys 0.020s
I20260807 19:48:48.386060 9106 raft_consensus.cc:359] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.382767 9099 tablet_bootstrap.cc:492] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.386844 9106 raft_consensus.cc:385] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.387270 9200 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:48.387293 9106 raft_consensus.cc:740] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.388490 9106 consensus_queue.cc:260] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.391260 9106 ts_tablet_manager.cc:1434] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:48:48.392290 9106 tablet_bootstrap.cc:492] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.393850 8632 catalog_manager.cc:5697] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:48.402777 9099 tablet_bootstrap.cc:654] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.413587 9156 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:48.429426 9220 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.431979 9101 tablet_bootstrap.cc:492] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.434938 9101 ts_tablet_manager.cc:1403] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.134s user 0.010s sys 0.014s
I20260807 19:48:48.438954 9106 tablet_bootstrap.cc:654] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.438213 9200 raft_consensus.cc:697] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:48.442267 9101 raft_consensus.cc:359] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.443657 9101 raft_consensus.cc:385] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.444034 9101 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.445514 9101 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.448468 9101 ts_tablet_manager.cc:1434] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.013s user 0.007s sys 0.001s
I20260807 19:48:48.449301 9101 tablet_bootstrap.cc:492] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.453840 9200 consensus_queue.cc:237] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.462184 9101 tablet_bootstrap.cc:654] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.487996 9106 tablet_bootstrap.cc:492] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.488597 9106 ts_tablet_manager.cc:1403] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.005s
I20260807 19:48:48.491155 9099 tablet_bootstrap.cc:492] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.491623 9099 ts_tablet_manager.cc:1403] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.109s user 0.013s sys 0.001s
I20260807 19:48:48.491834 9106 raft_consensus.cc:359] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.494508 9106 raft_consensus.cc:385] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.494860 9106 raft_consensus.cc:740] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.495666 9106 consensus_queue.cc:260] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.498418 9106 ts_tablet_manager.cc:1434] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20260807 19:48:48.499657 9099 raft_consensus.cc:359] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.500083 9106 tablet_bootstrap.cc:492] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.500805 9099 raft_consensus.cc:385] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.501502 9099 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.502746 9099 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.509466 9220 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.510643 9220 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.513051 9220 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.514988 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.521032 9099 ts_tablet_manager.cc:1434] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.029s user 0.004s sys 0.004s
I20260807 19:48:48.521517 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.522228 9021 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:48.522384 9099 tablet_bootstrap.cc:492] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.524427 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:48.525399 9117 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:48.525821 9117 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:48.526221 9117 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:48.528774 8740 leader_election.cc:343] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.529623 8632 catalog_manager.cc:5697] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:48.533749 9239 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:48.541622 9117 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.542361 9101 tablet_bootstrap.cc:492] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.542871 9101 ts_tablet_manager.cc:1403] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.094s user 0.031s sys 0.041s
I20260807 19:48:48.544188 9117 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.549595 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:48.551185 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:48.551868 9021 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.552186 9106 tablet_bootstrap.cc:654] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.554392 9099 tablet_bootstrap.cc:654] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.557344 9101 raft_consensus.cc:359] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.558105 9101 raft_consensus.cc:385] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.558441 9101 raft_consensus.cc:740] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.559232 9101 consensus_queue.cc:260] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.562868 8913 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.566601 9021 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:48.572158 9239 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.572256 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:48.573395 9220 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:48.573920 9220 raft_consensus.cc:697] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:48.574865 9220 consensus_queue.cc:237] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.578249 8913 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:48.567860 9101 ts_tablet_manager.cc:1434] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.025s user 0.002s sys 0.005s
I20260807 19:48:48.581224 9101 tablet_bootstrap.cc:492] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.587394 9099 tablet_bootstrap.cc:492] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.587934 9099 ts_tablet_manager.cc:1403] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.066s user 0.017s sys 0.006s
I20260807 19:48:48.590857 9099 raft_consensus.cc:359] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.591571 9099 raft_consensus.cc:385] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.591892 9099 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.592842 9099 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.596753 9106 tablet_bootstrap.cc:492] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.597308 9106 ts_tablet_manager.cc:1403] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.019s
I20260807 19:48:48.600147 9106 raft_consensus.cc:359] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.600849 9106 raft_consensus.cc:385] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.601107 9106 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.601698 9106 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.602246 9101 tablet_bootstrap.cc:654] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.595288 9099 ts_tablet_manager.cc:1434] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 19:48:48.603412 9099 tablet_bootstrap.cc:492] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.606145 9106 ts_tablet_manager.cc:1434] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:48:48.608951 9106 tablet_bootstrap.cc:492] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.616539 9220 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.616245 8629 catalog_manager.cc:5697] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:48.620476 9099 tablet_bootstrap.cc:654] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.626121 9106 tablet_bootstrap.cc:654] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.648428 9101 tablet_bootstrap.cc:492] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.648991 9101 ts_tablet_manager.cc:1403] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.068s user 0.014s sys 0.015s
I20260807 19:48:48.651868 9101 raft_consensus.cc:359] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.652684 9101 raft_consensus.cc:385] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.653082 9101 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.653913 9101 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.654574 9099 tablet_bootstrap.cc:492] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.655074 9099 ts_tablet_manager.cc:1403] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.001s
I20260807 19:48:48.657487 9099 raft_consensus.cc:359] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.657979 9099 raft_consensus.cc:385] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.658136 9099 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.658592 9099 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.660074 9101 ts_tablet_manager.cc:1434] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260807 19:48:48.660691 9099 ts_tablet_manager.cc:1434] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260807 19:48:48.661737 9099 tablet_bootstrap.cc:492] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.662864 9101 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.661772 9106 tablet_bootstrap.cc:492] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.664193 9106 ts_tablet_manager.cc:1403] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.055s user 0.005s sys 0.028s
I20260807 19:48:48.668709 9106 raft_consensus.cc:359] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.669456 9106 raft_consensus.cc:385] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.669739 9106 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.670466 9106 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.672916 9106 ts_tablet_manager.cc:1434] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 19:48:48.673849 9099 tablet_bootstrap.cc:654] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.673909 9106 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.675000 9239 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.675477 9239 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.678440 9145 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.678620 9101 tablet_bootstrap.cc:654] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.679011 9145 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.683022 9239 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:48.684298 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:48.684742 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.685035 9145 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:48.684703 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:48.687427 8789 raft_consensus.cc:2393] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:48.689453 8968 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:48.690616 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.695421 8972 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:48.696190 9145 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.697578 9099 tablet_bootstrap.cc:492] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.698067 9099 ts_tablet_manager.cc:1403] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.037s user 0.012s sys 0.009s
I20260807 19:48:48.699719 9239 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.701869 9021 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.705255 9145 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.711149 9107 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:48.713047 9106 tablet_bootstrap.cc:654] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.716260 9099 raft_consensus.cc:359] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.717020 9099 raft_consensus.cc:385] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.717294 9099 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.721016 9101 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.721573 9101 ts_tablet_manager.cc:1403] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.059s user 0.012s sys 0.000s
I20260807 19:48:48.717934 9099 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.729652 9099 ts_tablet_manager.cc:1434] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.031s user 0.007s sys 0.003s
I20260807 19:48:48.727672 9101 raft_consensus.cc:359] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.734095 9101 raft_consensus.cc:385] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.734405 9101 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.734668 9099 tablet_bootstrap.cc:492] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.735224 9101 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.737424 9101 ts_tablet_manager.cc:1434] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260807 19:48:48.737558 9273 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.738392 9101 tablet_bootstrap.cc:492] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.738123 9273 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.732924 9106 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.740489 9106 ts_tablet_manager.cc:1403] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.067s user 0.006s sys 0.017s
I20260807 19:48:48.743453 9106 raft_consensus.cc:359] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.744133 9106 raft_consensus.cc:385] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.744437 9106 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.745240 9106 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.745999 9101 tablet_bootstrap.cc:654] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.746084 9099 tablet_bootstrap.cc:654] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.747648 9106 ts_tablet_manager.cc:1434] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:48:48.749410 9106 tablet_bootstrap.cc:492] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.749584 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:48.750332 8789 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:48.757678 9101 tablet_bootstrap.cc:492] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.758219 9101 ts_tablet_manager.cc:1403] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20260807 19:48:48.759536 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:48.760672 9200 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:48.761055 9200 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:48.761505 9200 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.762578 9106 tablet_bootstrap.cc:654] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.765745 9273 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.766963 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:48.768972 8856 leader_election.cc:343] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.769187 9099 tablet_bootstrap.cc:492] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.769802 9099 ts_tablet_manager.cc:1403] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.035s user 0.016s sys 0.005s
I20260807 19:48:48.770438 9101 raft_consensus.cc:359] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.771205 9101 raft_consensus.cc:385] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.771494 9101 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.773131 9099 raft_consensus.cc:359] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.774003 9099 raft_consensus.cc:385] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.773895 9101 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.774544 9099 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.775333 9099 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.776731 9107 consensus_queue.cc:1048] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.790975 9099 ts_tablet_manager.cc:1434] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260807 19:48:48.798918 9099 tablet_bootstrap.cc:492] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.801923 9192 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.802462 9192 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.808821 9107 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.800557 9101 ts_tablet_manager.cc:1434] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.042s user 0.008s sys 0.004s
I20260807 19:48:48.809600 9192 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.810586 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.812326 9101 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.813937 9200 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.818300 9200 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.819939 9106 tablet_bootstrap.cc:492] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.820525 9106 ts_tablet_manager.cc:1403] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.024s
I20260807 19:48:48.812368 9107 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.825429 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:48.827442 9107 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.827865 9220 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.828397 9220 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.830866 9099 tablet_bootstrap.cc:654] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.831148 9220 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.828203 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:48.834250 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:48.833811 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:48.835973 8789 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.839084 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.841113 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:48:48.842924 8740 leader_election.cc:343] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.843493 8789 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:48.845299 9200 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.850737 9101 tablet_bootstrap.cc:654] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.853135 8913 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:48.854220 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.859221 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:48.859917 8856 leader_election.cc:343] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.862072 9106 raft_consensus.cc:359] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.874980 9106 raft_consensus.cc:385] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.875325 9106 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.876032 9106 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.877789 9099 tablet_bootstrap.cc:492] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.878281 9099 ts_tablet_manager.cc:1403] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.086s user 0.014s sys 0.018s
I20260807 19:48:48.878952 9106 ts_tablet_manager.cc:1434] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.058s user 0.005s sys 0.001s
I20260807 19:48:48.879927 9106 tablet_bootstrap.cc:492] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
W20260807 19:48:48.866065 8740 leader_election.cc:343] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.880728 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.881212 9099 raft_consensus.cc:359] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.881932 9099 raft_consensus.cc:385] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.882198 9099 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.882582 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.883448 9200 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:48.884023 9200 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:48.885008 9200 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.883687 9220 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:48:48.888293 8740 leader_election.cc:343] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:48.889261 8740 leader_election.cc:343] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.889719 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.882900 9099 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.890618 9137 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.892660 9099 ts_tablet_manager.cc:1434] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260807 19:48:48.894604 9137 consensus_queue.cc:1048] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:48.895155 9099 tablet_bootstrap.cc:492] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.903841 9106 tablet_bootstrap.cc:654] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.906265 8632 catalog_manager.cc:5697] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:48.913579 9099 tablet_bootstrap.cc:654] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:48.921664 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:48.931648 9137 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:48.932224 9137 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.935186 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:48.935796 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:48.936559 8740 leader_election.cc:343] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:48.937768 8740 leader_election.cc:343] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:48.938199 8740 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:48.934293 9137 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:48.939031 9220 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:48.940536 9192 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:48.941543 9101 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.942059 9101 ts_tablet_manager.cc:1403] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.130s user 0.018s sys 0.002s
I20260807 19:48:48.947140 9101 raft_consensus.cc:359] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.950783 9099 tablet_bootstrap.cc:492] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:48.947995 9101 raft_consensus.cc:385] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.951287 9099 ts_tablet_manager.cc:1403] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.056s user 0.012s sys 0.005s
I20260807 19:48:48.951558 9101 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.954180 9099 raft_consensus.cc:359] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.954306 9101 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.954926 9099 raft_consensus.cc:385] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.955245 9099 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.956004 9101 ts_tablet_manager.cc:1434] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260807 19:48:48.955921 9099 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.958331 9099 ts_tablet_manager.cc:1434] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:48:48.959311 9099 tablet_bootstrap.cc:492] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:48.960089 9276 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:48.963267 9106 tablet_bootstrap.cc:492] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:48.963781 9106 ts_tablet_manager.cc:1403] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.011s
I20260807 19:48:48.966022 9099 tablet_bootstrap.cc:654] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.966450 9106 raft_consensus.cc:359] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.967118 9106 raft_consensus.cc:385] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.967403 9106 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:48.959323 9101 tablet_bootstrap.cc:492] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:48.968308 9106 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.974948 9101 tablet_bootstrap.cc:654] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.985080 9106 ts_tablet_manager.cc:1434] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.021s user 0.000s sys 0.008s
I20260807 19:48:48.986044 9106 tablet_bootstrap.cc:492] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:48.989240 9101 tablet_bootstrap.cc:492] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:48.989732 9101 ts_tablet_manager.cc:1403] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.031s user 0.012s sys 0.000s
I20260807 19:48:48.991590 9106 tablet_bootstrap.cc:654] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:48.992694 9101 raft_consensus.cc:359] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:48.993429 9101 raft_consensus.cc:385] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:48.993711 9101 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.000387 9101 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.003954 9099 tablet_bootstrap.cc:492] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.004410 9099 ts_tablet_manager.cc:1403] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.045s user 0.014s sys 0.007s
I20260807 19:48:49.005728 9101 ts_tablet_manager.cc:1434] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260807 19:48:49.006690 9101 tablet_bootstrap.cc:492] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.007218 9099 raft_consensus.cc:359] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.007937 9099 raft_consensus.cc:385] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.008176 9099 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.008868 9099 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.013976 9101 tablet_bootstrap.cc:654] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.015211 9106 tablet_bootstrap.cc:492] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.015724 9106 ts_tablet_manager.cc:1403] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.005s
I20260807 19:48:49.014066 9099 ts_tablet_manager.cc:1434] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260807 19:48:49.017160 9099 tablet_bootstrap.cc:492] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.018806 9106 raft_consensus.cc:359] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.019587 9106 raft_consensus.cc:385] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.019927 9106 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.020732 9106 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.022822 9106 ts_tablet_manager.cc:1434] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260807 19:48:49.023460 9145 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.023983 9145 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.024966 9106 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.026355 9099 tablet_bootstrap.cc:654] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.027221 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.027920 8789 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.030103 8968 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.031402 9239 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.031635 9220 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.032202 9220 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.033718 9106 tablet_bootstrap.cc:654] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.034205 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.034425 9220 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.034875 8915 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.037220 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:49.039109 8740 leader_election.cc:343] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.040110 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:49.041899 8740 leader_election.cc:343] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.042404 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.043835 9220 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.044648 9239 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.045099 9239 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.046860 9101 tablet_bootstrap.cc:492] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.047375 9101 ts_tablet_manager.cc:1403] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.041s user 0.017s sys 0.006s
I20260807 19:48:49.052536 9145 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.056787 9239 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.059267 9099 tablet_bootstrap.cc:492] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.059860 9099 ts_tablet_manager.cc:1403] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.003s
I20260807 19:48:49.059770 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.060873 8789 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.061586 9101 raft_consensus.cc:359] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.062263 9101 raft_consensus.cc:385] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.062613 9101 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.062322 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.063314 8915 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.063541 9101 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.064096 9220 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.064580 9192 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.065186 9192 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.064639 9220 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.066475 9239 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.068678 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.069510 8788 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.070530 9220 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.067796 9192 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.071619 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.073289 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.073944 9020 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:49.075153 8849 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:49.075415 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.076875 9106 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.077368 9106 ts_tablet_manager.cc:1403] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.053s user 0.021s sys 0.007s
I20260807 19:48:49.077396 9099 raft_consensus.cc:359] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.077483 9273 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.078465 9099 raft_consensus.cc:385] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.078745 9273 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.078904 9099 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.078967 9276 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.079867 9099 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.079622 9276 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.079258 9273 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:49.081140 8856 leader_election.cc:343] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.082382 9276 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.082553 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.083240 8913 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:49.084844 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.085500 9020 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.086776 8856 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.088076 9192 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.088474 9192 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.088819 9192 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.090168 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.090811 8790 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.093205 8789 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.094811 8968 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.095232 9101 ts_tablet_manager.cc:1434] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.048s user 0.004s sys 0.004s
I20260807 19:48:49.097034 9101 tablet_bootstrap.cc:492] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.084149 9220 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.100803 9220 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.101235 9220 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.097776 9099 ts_tablet_manager.cc:1434] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260807 19:48:49.104004 9099 tablet_bootstrap.cc:492] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.095422 9192 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.100737 9239 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.109364 9137 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.109485 9192 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.109916 9137 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.112185 9137 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.113907 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.113747 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.102700 8915 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.114593 9106 raft_consensus.cc:359] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.114761 8789 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.115922 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:48:49.116169 8740 leader_election.cc:343] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.116750 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.117506 9220 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.120024 9220 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.115314 9106 raft_consensus.cc:385] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.121031 9106 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.121204 9101 tablet_bootstrap.cc:654] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.122713 9106 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:49.124204 8740 leader_election.cc:343] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.124424 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.125034 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:49.125276 8915 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.125748 9020 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.128031 9101 tablet_bootstrap.cc:492] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.128608 9101 ts_tablet_manager.cc:1403] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.002s
I20260807 19:48:49.129669 9027 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.125000 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.133569 9137 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.134775 9020 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:49.136549 9099 tablet_bootstrap.cc:654] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.137533 9137 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.138036 9101 raft_consensus.cc:359] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.138127 9137 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.140552 9137 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.141073 9101 raft_consensus.cc:385] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.141381 9101 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.141603 8789 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:49.142025 9101 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.143455 8915 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:49.144505 9101 ts_tablet_manager.cc:1434] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20260807 19:48:49.145642 9101 tablet_bootstrap.cc:492] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.146528 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.148097 9239 raft_consensus.cc:697] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:49.148900 9239 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.150508 9027 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:49.151286 9146 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.151789 9146 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.152036 8856 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.152920 9192 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.153579 9192 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:49.153985 9146 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.154443 9192 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.154940 9273 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.150521 9106 ts_tablet_manager.cc:1434] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.073s user 0.005s sys 0.003s
I20260807 19:48:49.152968 9145 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.156529 9106 tablet_bootstrap.cc:492] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.156594 9145 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.157244 9099 tablet_bootstrap.cc:492] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.157732 9099 ts_tablet_manager.cc:1403] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.054s user 0.009s sys 0.006s
I20260807 19:48:49.159973 9101 tablet_bootstrap.cc:654] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.155166 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.165468 8789 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.165550 9107 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.166648 9107 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:49.166615 9099 raft_consensus.cc:359] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.167514 9099 raft_consensus.cc:385] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.167809 9099 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.167618 9107 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.169185 9099 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.167640 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.171712 9099 ts_tablet_manager.cc:1434] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.014s user 0.003s sys 0.004s
I20260807 19:48:49.172734 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:48:49.173216 8740 leader_election.cc:343] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.173084 9220 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:49.173969 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.175603 9146 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.176033 9146 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.175740 9099 tablet_bootstrap.cc:492] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.176486 9146 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.181916 8790 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:49.185807 9220 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.186435 9220 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.189330 9145 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.164072 9106 tablet_bootstrap.cc:654] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.192159 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.193118 8915 raft_consensus.cc:2393] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:49.218293 9276 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.219030 9276 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.221449 9146 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.223341 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.223783 9146 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.227965 9099 tablet_bootstrap.cc:654] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.225485 9276 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.229372 9220 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.235178 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:49.232991 8856 leader_election.cc:343] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.240752 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.243798 8632 catalog_manager.cc:5697] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:49.244374 9273 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.247771 8790 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.249075 9106 tablet_bootstrap.cc:492] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.249501 9106 ts_tablet_manager.cc:1403] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.093s user 0.012s sys 0.002s
I20260807 19:48:49.254792 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:49.254415 9106 raft_consensus.cc:359] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.255635 9020 raft_consensus.cc:2393] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:49.255857 9106 raft_consensus.cc:385] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.256476 8790 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.241779 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.257800 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.256197 9106 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.259078 9106 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.259176 9145 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.260673 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.265146 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.253135 8628 catalog_manager.cc:5697] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:49.267001 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.266675 8632 catalog_manager.cc:5697] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:49.262768 9101 tablet_bootstrap.cc:492] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.263063 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.267639 8790 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.268978 8911 raft_consensus.cc:2393] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:49.269801 9145 raft_consensus.cc:697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:49.270809 9145 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.271435 9106 ts_tablet_manager.cc:1434] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.022s user 0.007s sys 0.000s
I20260807 19:48:49.272470 9106 tablet_bootstrap.cc:492] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.268244 9101 ts_tablet_manager.cc:1403] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.123s user 0.021s sys 0.037s
I20260807 19:48:49.274425 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.276211 8789 raft_consensus.cc:2393] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:49.283471 9101 raft_consensus.cc:359] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.284334 9101 raft_consensus.cc:385] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.284148 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:49.284915 9101 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.285661 9101 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:49.285920 8740 leader_election.cc:343] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:49.287173 8740 leader_election.cc:343] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.287034 9137 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:49.287631 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.286018 9145 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.288520 9101 ts_tablet_manager.cc:1434] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20260807 19:48:49.289554 9101 tablet_bootstrap.cc:492] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.291321 9106 tablet_bootstrap.cc:654] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:49.292867 8740 leader_election.cc:343] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.293195 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.294124 9107 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.297358 9099 tablet_bootstrap.cc:492] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.298739 9099 ts_tablet_manager.cc:1403] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.123s user 0.007s sys 0.006s
I20260807 19:48:49.299201 8632 catalog_manager.cc:5697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:49.300536 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.294222 9137 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.307207 9273 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.307642 8849 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.311200 9101 tablet_bootstrap.cc:654] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.312161 9192 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.302695 9145 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.307643 9273 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.319275 9273 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.329454 9099 raft_consensus.cc:359] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.337299 9099 raft_consensus.cc:385] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.320195 9145 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.337805 9099 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.339286 9099 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.340719 9145 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.345791 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.346751 8789 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.348451 8968 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.349428 9146 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.349829 9146 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.350163 9146 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.351470 9099 ts_tablet_manager.cc:1434] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.038s user 0.005s sys 0.002s
I20260807 19:48:49.356472 9273 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.357709 9099 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.352855 9145 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.362425 9273 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.363770 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:49.364765 9021 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.367507 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.368263 8911 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.370957 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.378505 9106 tablet_bootstrap.cc:492] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.379169 9106 ts_tablet_manager.cc:1403] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.107s user 0.019s sys 0.013s
I20260807 19:48:49.380263 8790 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.381448 9099 tablet_bootstrap.cc:654] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.381812 9106 raft_consensus.cc:359] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.382354 9106 raft_consensus.cc:385] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.382570 9106 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.383109 9106 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.390128 9101 tablet_bootstrap.cc:492] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.390678 9101 ts_tablet_manager.cc:1403] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.101s user 0.009s sys 0.005s
I20260807 19:48:49.390120 9106 ts_tablet_manager.cc:1434] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:48:49.393551 9106 tablet_bootstrap.cc:492] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.394966 9101 raft_consensus.cc:359] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.395712 9101 raft_consensus.cc:385] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.396025 9101 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.396715 9101 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.399720 9099 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.400287 9099 ts_tablet_manager.cc:1403] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.047s user 0.006s sys 0.005s
I20260807 19:48:49.402070 9101 ts_tablet_manager.cc:1434] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260807 19:48:49.403115 9101 tablet_bootstrap.cc:492] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.406369 9192 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:49.368301 9146 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.418151 9099 raft_consensus.cc:359] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.419795 9099 raft_consensus.cc:385] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.420077 9099 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.420748 9099 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.422415 9099 ts_tablet_manager.cc:1434] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260807 19:48:49.423650 9099 tablet_bootstrap.cc:492] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.428449 9021 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:49.430102 8856 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.432379 9273 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.432971 9273 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:49.433104 9200 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:49.433938 9273 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.443125 9106 tablet_bootstrap.cc:654] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.445046 9146 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.446163 8790 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:49.446834 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.447574 8911 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.447984 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.451929 9101 tablet_bootstrap.cc:654] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.454553 8789 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.468971 8789 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.469066 8911 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.470584 8972 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.471495 9146 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.471930 9146 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:49.473665 9146 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.481484 8629 catalog_manager.cc:5697] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:49.507102 9099 tablet_bootstrap.cc:654] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.508240 8628 catalog_manager.cc:5697] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:49.520838 9101 tablet_bootstrap.cc:492] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.521363 9101 ts_tablet_manager.cc:1403] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.118s user 0.009s sys 0.005s
I20260807 19:48:49.524263 9101 raft_consensus.cc:359] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.525002 9101 raft_consensus.cc:385] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.525271 9101 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.526201 9101 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.527438 9106 tablet_bootstrap.cc:492] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.527944 9106 ts_tablet_manager.cc:1403] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.135s user 0.022s sys 0.009s
I20260807 19:48:49.530818 9106 raft_consensus.cc:359] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.531464 9106 raft_consensus.cc:385] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.531716 9106 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.532400 9106 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.533105 9099 tablet_bootstrap.cc:492] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.533617 9099 ts_tablet_manager.cc:1403] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.110s user 0.011s sys 0.007s
I20260807 19:48:49.534368 9106 ts_tablet_manager.cc:1434] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 19:48:49.537101 9101 ts_tablet_manager.cc:1434] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260807 19:48:49.537889 9101 tablet_bootstrap.cc:492] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.541164 9106 tablet_bootstrap.cc:492] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.543754 9101 tablet_bootstrap.cc:654] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.536166 9099 raft_consensus.cc:359] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.544564 9099 raft_consensus.cc:385] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.544787 9099 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.545392 9099 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.549755 9101 tablet_bootstrap.cc:492] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.550348 9101 ts_tablet_manager.cc:1403] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260807 19:48:49.552934 9101 raft_consensus.cc:359] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.553612 9101 raft_consensus.cc:385] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.553885 9101 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.554543 9101 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.555123 9106 tablet_bootstrap.cc:654] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.557214 9101 ts_tablet_manager.cc:1434] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260807 19:48:49.559233 9101 tablet_bootstrap.cc:492] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.561964 9099 ts_tablet_manager.cc:1434] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.028s user 0.006s sys 0.000s
I20260807 19:48:49.562986 9099 tablet_bootstrap.cc:492] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.565229 9101 tablet_bootstrap.cc:654] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.572291 9099 tablet_bootstrap.cc:654] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.573628 9106 tablet_bootstrap.cc:492] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.574108 9106 ts_tablet_manager.cc:1403] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.012s
I20260807 19:48:49.576442 9220 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.576880 9106 raft_consensus.cc:359] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.576988 9220 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.577636 9106 raft_consensus.cc:385] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.577956 9106 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.578752 9106 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.579372 9220 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.579977 9220 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.582981 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.585384 9106 ts_tablet_manager.cc:1434] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260807 19:48:49.586367 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.586378 9106 tablet_bootstrap.cc:492] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
W20260807 19:48:49.587881 8740 leader_election.cc:343] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.587958 9146 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:48:49.588662 8740 leader_election.cc:343] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.588541 9146 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.589056 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.589601 9220 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.590708 9137 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.590871 9146 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.592108 9137 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:49.592145 9220 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.596459 9101 tablet_bootstrap.cc:492] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.596964 9101 ts_tablet_manager.cc:1403] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.038s user 0.018s sys 0.011s
I20260807 19:48:49.596772 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.597429 8789 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.598459 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.599094 8913 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.600811 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.595808 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.601868 8911 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:49.602461 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.603103 9021 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:49.604547 9145 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.604947 9145 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.605279 9145 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.606386 9106 tablet_bootstrap.cc:654] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.606752 9099 tablet_bootstrap.cc:492] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.607228 9099 ts_tablet_manager.cc:1403] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.044s user 0.013s sys 0.002s
I20260807 19:48:49.610538 9099 raft_consensus.cc:359] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.611230 9099 raft_consensus.cc:385] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.611903 9099 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.612680 9099 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.610905 9101 raft_consensus.cc:359] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.613488 9101 raft_consensus.cc:385] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.613803 9101 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.614688 9099 ts_tablet_manager.cc:1434] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260807 19:48:49.614513 9101 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.615618 9099 tablet_bootstrap.cc:492] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.622835 9101 ts_tablet_manager.cc:1434] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.015s user 0.002s sys 0.004s
I20260807 19:48:49.623989 9101 tablet_bootstrap.cc:492] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.621337 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.629465 9220 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.629959 9220 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.630311 9220 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.626835 9273 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.632875 9273 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.634089 9145 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.635552 9200 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:49.637008 9200 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.638398 9200 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.639407 9220 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.637034 9145 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.645026 9146 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.647656 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.650436 9146 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.655236 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.655784 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:49.657467 8856 leader_election.cc:343] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.659842 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.660266 9022 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.660483 8788 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.660832 9146 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.663177 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.663834 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.664206 9192 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.664773 9192 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.665131 9192 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.652465 9200 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.645171 9273 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.656004 8911 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.668866 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.669759 8787 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.671298 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.673220 8789 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.677412 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.678189 8789 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.678720 8788 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.679611 9192 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.680778 9146 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:49.685420 8849 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.680932 8968 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.687922 9145 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.688300 9145 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.688748 9145 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.689931 9273 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.690382 9273 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.690703 9273 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.694415 9106 tablet_bootstrap.cc:492] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.694939 9106 ts_tablet_manager.cc:1403] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.109s user 0.018s sys 0.007s
I20260807 19:48:49.684322 8911 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.696903 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.697520 8915 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.697458 8968 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.698764 9106 raft_consensus.cc:359] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.699744 9106 raft_consensus.cc:385] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.699844 9146 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.700784 9146 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:49.700100 9106 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.699090 9099 tablet_bootstrap.cc:654] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.702842 9106 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.704104 8913 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:49.705300 9106 ts_tablet_manager.cc:1434] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.010s user 0.002s sys 0.005s
I20260807 19:48:49.707408 9106 tablet_bootstrap.cc:492] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.708971 9146 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.712481 9220 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.715613 9145 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.719023 8915 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:49.720294 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:49.721532 9137 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.722007 9137 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:49.723172 9137 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.723685 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.724511 8787 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.726814 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:49.727454 9020 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.725582 9145 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.735795 9192 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.722997 9273 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.753509 9101 tablet_bootstrap.cc:654] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.728482 9117 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260807 19:48:49.735133 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:49.751372 9145 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.767544 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:49.768482 9027 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:49.769644 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.774607 8911 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:49.780580 9106 tablet_bootstrap.cc:654] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.782202 8628 catalog_manager.cc:5697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:49.784837 9276 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:49.786724 9276 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.787227 9276 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.791739 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.765435 9145 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.795249 9192 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.799860 9273 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.802634 9276 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.805961 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:49.806569 9145 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:49.806823 9022 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.799909 9351 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260807 19:48:49.819299 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.820039 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.820057 8789 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:49.820591 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.821074 8911 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:49.822017 8968 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.832742 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:49.833640 8783 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:49.826623 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:49.837291 9239 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:49.837699 8789 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:49.840153 8856 leader_election.cc:343] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.845248 8849 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.846567 9276 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:49.846971 9276 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:49.847283 9276 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:49.848404 9099 tablet_bootstrap.cc:492] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:49.848917 9099 ts_tablet_manager.cc:1403] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.234s user 0.016s sys 0.018s
I20260807 19:48:49.864735 8787 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:49.885778 9390 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:49.886379 9390 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.892696 9390 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.896210 9020 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:49.900197 8968 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:49.905521 8629 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:49.909581 9099 raft_consensus.cc:359] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.910373 9099 raft_consensus.cc:385] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.910661 9099 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.915941 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:49.906339 9145 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.917963 9145 raft_consensus.cc:697] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:49.919247 9106 tablet_bootstrap.cc:492] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:49.918915 9145 consensus_queue.cc:237] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.919750 9106 ts_tablet_manager.cc:1403] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.213s user 0.027s sys 0.015s
I20260807 19:48:49.919441 9099 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:49.921983 8740 leader_election.cc:343] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.923228 9106 raft_consensus.cc:359] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.923942 9106 raft_consensus.cc:385] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.924197 9106 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.924432 9101 tablet_bootstrap.cc:492] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:49.924901 9106 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.926580 9099 ts_tablet_manager.cc:1434] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.077s user 0.007s sys 0.001s
I20260807 19:48:49.927569 9099 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:49.928658 9101 ts_tablet_manager.cc:1403] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.305s user 0.021s sys 0.017s
I20260807 19:48:49.928848 9106 ts_tablet_manager.cc:1434] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 19:48:49.930565 8789 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:49.931039 9106 tablet_bootstrap.cc:492] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:49.937759 9101 raft_consensus.cc:359] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.941396 9101 raft_consensus.cc:385] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:49.941778 9101 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:49.942492 9101 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.944665 9101 ts_tablet_manager.cc:1434] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260807 19:48:49.946408 9099 tablet_bootstrap.cc:654] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:49.954687 9022 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:49.912380 9276 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.956188 9146 raft_consensus.cc:2764] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260807 19:48:49.958884 8856 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:49.960086 9200 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:49.960597 9200 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:49.961462 9200 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.962616 9276 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:49.956586 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:49.967092 9390 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:48:49.967204 8740 leader_election.cc:343] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:49.967901 8740 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:49.967798 9390 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:49.975649 9388 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:49.986711 9101 tablet_bootstrap.cc:492] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:49.996469 8629 catalog_manager.cc:5697] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:49.997848 9390 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.008925 9106 tablet_bootstrap.cc:654] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.037175 9390 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.047158 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.047318 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:50.013068 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:50.016988 9145 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.112138 9192 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:48:50.121281 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 45125.2us
I20260807 19:48:50.123543 9192 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.125772 9192 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.128100 9145 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.013338 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.137709 9145 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:50.142153 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:50.145022 8856 leader_election.cc:343] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.145516 9099 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:50.146090 9099 ts_tablet_manager.cc:1403] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.219s user 0.023s sys 0.028s
W20260807 19:48:50.151742 8740 leader_election.cc:343] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.102349 9351 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260807 19:48:50.139613 9389 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260807 19:48:50.157967 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:50.161639 9409 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.138796 8788 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.139021 9023 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.169162 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.170243 8915 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:50.122529 9390 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.183810 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.192443 9276 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.192764 8787 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
W20260807 19:48:50.197253 8740 leader_election.cc:343] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.200906 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.203301 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.206166 9388 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.207381 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.208544 9101 tablet_bootstrap.cc:654] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.208284 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:50.214980 8740 leader_election.cc:343] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.217737 9099 raft_consensus.cc:359] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.218487 9099 raft_consensus.cc:385] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.218770 9099 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.219763 9099 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.204754 9137 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.226812 9137 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.227303 9137 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.221421 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.225925 9414 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:50.209337 9390 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.224169 8788 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
W20260807 19:48:50.234894 8740 leader_election.cc:343] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.235368 8740 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.236428 9023 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:50.239404 8849 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:50.236109 9117 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.249073 9099 ts_tablet_manager.cc:1434] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.103s user 0.007s sys 0.000s
I20260807 19:48:50.250294 9099 tablet_bootstrap.cc:492] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:50.251829 9200 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:50.233609 9414 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:50.253346 9200 raft_consensus.cc:697] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:50.253909 9414 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.229557 8968 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:50.258960 9413 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.053s
I20260807 19:48:50.262468 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.263587 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.254310 9200 consensus_queue.cc:237] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.253376 9137 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:48:50.264966 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 49424.3us
W20260807 19:48:50.270237 8740 leader_election.cc:343] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.282682 9412 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:48:50.291380 8740 leader_election.cc:343] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.291953 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.292743 9388 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.302906 8629 catalog_manager.cc:5697] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:50.313258 9106 tablet_bootstrap.cc:492] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:50.313802 9106 ts_tablet_manager.cc:1403] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.383s user 0.009s sys 0.021s
I20260807 19:48:50.318630 9415 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 19:48:50.312687 9390 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.319928 9390 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.321018 9101 tablet_bootstrap.cc:492] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.321554 9101 ts_tablet_manager.cc:1403] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.335s user 0.019s sys 0.012s
I20260807 19:48:50.344105 9101 raft_consensus.cc:359] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.345082 9101 raft_consensus.cc:385] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.345376 9101 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.348472 9101 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.351029 9415 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.354955 9414 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.355958 9390 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.357815 9099 tablet_bootstrap.cc:654] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.338498 9137 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260807 19:48:50.363729 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.365113 9106 raft_consensus.cc:359] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.365831 9106 raft_consensus.cc:385] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.366082 9106 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.367141 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:50.371681 8788 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.373904 9101 ts_tablet_manager.cc:1434] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.052s user 0.012s sys 0.007s
I20260807 19:48:50.375213 9101 tablet_bootstrap.cc:492] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
W20260807 19:48:50.377434 8740 leader_election.cc:343] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.380821 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.379520 9106 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.394524 9414 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.399973 9409 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.400800 9106 ts_tablet_manager.cc:1434] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.087s user 0.008s sys 0.000s
I20260807 19:48:50.404626 9106 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
W20260807 19:48:50.405913 8740 leader_election.cc:343] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.406455 8740 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.411255 9117 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.421340 8788 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:50.427928 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:50.432787 9414 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:50.433445 9414 raft_consensus.cc:697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:50.434418 9414 consensus_queue.cc:237] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.453226 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:50.453976 9023 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.484220 8628 catalog_manager.cc:5697] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:50.499984 9409 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.544705 9273 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.545221 9273 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.547300 9273 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.549232 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.549937 8789 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:50.552858 9106 tablet_bootstrap.cc:654] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.567168 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:50.568264 9273 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:50.568711 9273 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:50.569135 9273 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.569525 9101 tablet_bootstrap.cc:654] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.567716 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.572003 9023 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
W20260807 19:48:50.574558 8856 leader_election.cc:343] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.580215 9273 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.587848 9099 tablet_bootstrap.cc:492] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:50.591337 9099 ts_tablet_manager.cc:1403] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.341s user 0.015s sys 0.047s
I20260807 19:48:50.594209 9099 raft_consensus.cc:359] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.594564 9192 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:50.594942 9099 raft_consensus.cc:385] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.595223 9099 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.595907 9099 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.598331 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:50.598531 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:50.599181 8780 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.599578 9099 ts_tablet_manager.cc:1434] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260807 19:48:50.600171 8856 leader_election.cc:343] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.600653 9099 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:50.600881 9273 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.604763 9137 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.605346 9137 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.607436 9137 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.610759 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.610880 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:50.612588 8740 leader_election.cc:343] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:50.613758 8740 leader_election.cc:343] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.614130 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.614989 9137 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.604830 9106 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:50.616189 9106 ts_tablet_manager.cc:1403] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.215s user 0.018s sys 0.007s
I20260807 19:48:50.618944 9106 raft_consensus.cc:359] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.619649 9106 raft_consensus.cc:385] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.620065 9106 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.623293 9273 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:50.623811 9099 tablet_bootstrap.cc:654] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.637048 9106 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.623984 9101 tablet_bootstrap.cc:492] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.639379 9101 ts_tablet_manager.cc:1403] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.264s user 0.011s sys 0.002s
I20260807 19:48:50.639937 9106 ts_tablet_manager.cc:1434] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260807 19:48:50.639537 8629 catalog_manager.cc:5697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:50.651392 9101 raft_consensus.cc:359] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.633160 8780 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:50.652383 9101 raft_consensus.cc:385] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.652769 9101 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.654023 9106 tablet_bootstrap.cc:492] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:50.653667 9101 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.655925 9101 ts_tablet_manager.cc:1434] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260807 19:48:50.656903 9101 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:50.658514 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.660822 9200 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:50.661348 9200 raft_consensus.cc:697] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:50.662210 9200 consensus_queue.cc:237] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.663091 9101 tablet_bootstrap.cc:654] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.670729 9106 tablet_bootstrap.cc:654] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.679085 9099 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:50.679601 9099 ts_tablet_manager.cc:1403] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.079s user 0.005s sys 0.005s
I20260807 19:48:50.681993 9101 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.682575 9101 ts_tablet_manager.cc:1403] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.006s
I20260807 19:48:50.682420 9099 raft_consensus.cc:359] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.683207 9099 raft_consensus.cc:385] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.683496 9099 raft_consensus.cc:740] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.684293 9099 consensus_queue.cc:260] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.685375 9101 raft_consensus.cc:359] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.686084 9101 raft_consensus.cc:385] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.686367 9101 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.686472 9099 ts_tablet_manager.cc:1434] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260807 19:48:50.687072 9101 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.690917 9101 ts_tablet_manager.cc:1434] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260807 19:48:50.691975 9101 tablet_bootstrap.cc:492] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:50.697736 9099 tablet_bootstrap.cc:492] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:50.700168 9106 tablet_bootstrap.cc:492] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:50.700796 9106 ts_tablet_manager.cc:1403] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.047s user 0.013s sys 0.000s
I20260807 19:48:50.706609 9106 raft_consensus.cc:359] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.707479 9106 raft_consensus.cc:385] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.707806 9106 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.703651 8632 catalog_manager.cc:5697] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:50.711933 9106 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.717710 9106 ts_tablet_manager.cc:1434] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260807 19:48:50.721315 9101 tablet_bootstrap.cc:654] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.725760 9099 tablet_bootstrap.cc:654] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.728078 9106 tablet_bootstrap.cc:492] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:50.739459 9106 tablet_bootstrap.cc:654] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.744656 9099 tablet_bootstrap.cc:492] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:50.745185 9099 ts_tablet_manager.cc:1403] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.048s user 0.011s sys 0.001s
I20260807 19:48:50.747895 9099 raft_consensus.cc:359] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.748673 9099 raft_consensus.cc:385] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.748970 9273 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.750103 9273 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.752302 9273 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.748963 9099 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.753051 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.753525 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:50.754552 8849 leader_election.cc:343] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:50.755136 8856 leader_election.cc:343] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.755578 8856 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.756916 9276 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.760488 9276 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:50.766533 9101 tablet_bootstrap.cc:492] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.767057 9101 ts_tablet_manager.cc:1403] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.075s user 0.016s sys 0.000s
I20260807 19:48:50.770426 9101 raft_consensus.cc:359] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.772200 9101 raft_consensus.cc:385] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.772536 9101 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.773365 9101 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.775017 9101 ts_tablet_manager.cc:1434] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:48:50.775985 9101 tablet_bootstrap.cc:492] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:50.753511 9099 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.785301 9276 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.785840 9276 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.787557 9099 ts_tablet_manager.cc:1434] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260807 19:48:50.787981 9276 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.788640 9099 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:50.789880 9101 tablet_bootstrap.cc:654] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.793995 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:48:50.795537 8849 leader_election.cc:343] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.798501 9106 tablet_bootstrap.cc:492] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:50.799010 9106 ts_tablet_manager.cc:1403] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.071s user 0.010s sys 0.000s
I20260807 19:48:50.803493 9106 raft_consensus.cc:359] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.804260 9106 raft_consensus.cc:385] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.806636 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.808768 9106 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.809973 9106 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.812016 9106 ts_tablet_manager.cc:1434] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260807 19:48:50.812989 9106 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:50.813344 9200 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:50.819937 9101 tablet_bootstrap.cc:492] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.820492 9101 ts_tablet_manager.cc:1403] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.004s
I20260807 19:48:50.823202 9101 raft_consensus.cc:359] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.823875 9101 raft_consensus.cc:385] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.824158 9101 raft_consensus.cc:740] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.827085 9101 consensus_queue.cc:260] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.828630 9099 tablet_bootstrap.cc:654] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.829629 9101 ts_tablet_manager.cc:1434] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:48:50.831434 9101 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:50.837570 9106 tablet_bootstrap.cc:654] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.839630 9101 tablet_bootstrap.cc:654] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:50.840526 8856 leader_election.cc:343] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.841408 8856 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.842788 9099 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:50.845974 9101 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.847491 9099 ts_tablet_manager.cc:1403] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.002s
I20260807 19:48:50.847500 9101 ts_tablet_manager.cc:1403] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.000s
I20260807 19:48:50.848714 9200 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.850579 9099 raft_consensus.cc:359] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.850692 9101 raft_consensus.cc:359] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.851358 9099 raft_consensus.cc:385] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.852121 9099 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.852128 9101 raft_consensus.cc:385] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.852720 9101 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.852995 9099 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.853626 9101 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.855196 9106 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:50.855697 9106 ts_tablet_manager.cc:1403] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.043s user 0.007s sys 0.005s
I20260807 19:48:50.858407 9106 raft_consensus.cc:359] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.859084 9106 raft_consensus.cc:385] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.859345 9106 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.861923 9101 ts_tablet_manager.cc:1434] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.014s user 0.005s sys 0.000s
I20260807 19:48:50.861953 9099 ts_tablet_manager.cc:1434] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260807 19:48:50.863257 9101 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:50.863281 9099 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:50.859989 9106 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.866587 9351 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.867146 9351 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.868634 9106 ts_tablet_manager.cc:1434] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260807 19:48:50.869436 9351 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:50.869732 9106 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:50.871351 9101 tablet_bootstrap.cc:654] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.871210 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.871963 8906 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:50.873550 8972 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:50.876449 9409 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:50.876681 9099 tablet_bootstrap.cc:654] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:50.875994 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.876889 9409 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:50.877264 9409 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.879184 9200 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.879666 9200 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.881243 8787 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:50.881809 9200 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:50.884491 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.885038 9106 tablet_bootstrap.cc:654] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:50.887450 8856 leader_election.cc:343] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.889631 9351 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.890251 9351 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.890882 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.891011 9273 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:50.892634 9351 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:48:50.893889 8849 leader_election.cc:343] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.894343 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.896164 9409 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.896116 9405 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.899890 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.900760 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.902590 8787 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:50.903038 9409 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:50.903345 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:50.904004 8780 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:50.904686 8972 leader_election.cc:343] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.906612 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:50.907614 9409 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:50.908098 9409 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:50.908545 9409 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.911578 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:50.912262 8906 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.915642 9405 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:50.918522 8780 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:50.919728 8968 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:50.922187 9413 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:50.928803 9351 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.929054 9413 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:50.929405 9351 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.931617 9351 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:50.933928 9200 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.934430 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:50.934475 9200 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.937659 9200 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:48:50.939116 8972 leader_election.cc:343] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.939775 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.940029 9413 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.942425 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:50.944156 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:50.945487 9101 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:50.942140 8779 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:50.945987 9101 ts_tablet_manager.cc:1403] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.083s user 0.008s sys 0.004s
I20260807 19:48:50.947070 9409 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:50.948927 8856 leader_election.cc:343] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:50.951682 8849 leader_election.cc:343] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.952108 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:50.953454 9200 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:50.955740 9409 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:50.955968 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:50.956651 8780 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.957090 8968 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:50.958797 9351 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:50.959277 9351 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:50.959712 9351 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:50.968200 8906 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:50.982442 9101 raft_consensus.cc:359] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.983388 8780 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:50.983851 9117 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:50.985622 9106 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:50.988441 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:50.989991 9106 ts_tablet_manager.cc:1403] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.120s user 0.024s sys 0.024s
I20260807 19:48:50.989734 9101 raft_consensus.cc:385] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:50.994423 9101 raft_consensus.cc:740] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:50.996178 9101 consensus_queue.cc:260] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:50.990900 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:48:50.997207 8972 leader_election.cc:343] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:50.987192 9099 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:50.997867 9099 ts_tablet_manager.cc:1403] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.135s user 0.024s sys 0.022s
I20260807 19:48:50.989457 9117 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.000044 9101 ts_tablet_manager.cc:1434] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.054s user 0.008s sys 0.000s
I20260807 19:48:51.000376 9409 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.001366 9409 raft_consensus.cc:697] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:51.002231 9409 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.005105 9137 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.005635 9137 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.006006 9117 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.007308 9351 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.009317 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.009580 9106 raft_consensus.cc:359] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.010268 9106 raft_consensus.cc:385] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.010556 9106 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
W20260807 19:48:51.010918 8740 leader_election.cc:343] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.007746 9137 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.013527 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.014894 9101 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
W20260807 19:48:51.015865 8740 leader_election.cc:343] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.016021 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.020583 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:51.020623 8740 leader_election.cc:343] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.021605 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.023869 9388 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:51.025923 9351 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.012141 9106 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.017426 9099 raft_consensus.cc:359] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.030048 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:51.030449 9099 raft_consensus.cc:385] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.030807 9099 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.031582 9099 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.033842 9101 tablet_bootstrap.cc:654] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.034915 8787 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.044727 8629 catalog_manager.cc:5697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:51.049217 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:48:51.050201 8740 leader_election.cc:343] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.030733 9106 ts_tablet_manager.cc:1434] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.037s user 0.004s sys 0.002s
I20260807 19:48:51.050661 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.051419 9106 tablet_bootstrap.cc:492] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:51.046239 9099 ts_tablet_manager.cc:1434] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.048s user 0.006s sys 0.000s
I20260807 19:48:51.051357 9388 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:51.052556 9099 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:51.059971 9137 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.060601 9137 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.045796 8787 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:51.063892 9101 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:51.064323 9101 ts_tablet_manager.cc:1403] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.050s user 0.007s sys 0.006s
I20260807 19:48:51.065239 8968 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:51.062642 9137 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.066089 9409 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.066578 9409 raft_consensus.cc:697] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:51.067435 9409 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.068070 9413 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.068413 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.069406 9413 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:51.070534 8740 leader_election.cc:343] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.071231 9099 tablet_bootstrap.cc:654] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.071621 9413 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.072850 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.073046 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.073740 8906 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:51.075120 9101 raft_consensus.cc:359] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.075868 9101 raft_consensus.cc:385] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.075579 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.078289 9137 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.079376 9137 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.076150 9101 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.079864 9137 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.080840 9101 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:51.081396 8968 leader_election.cc:343] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.078254 8632 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:48:51.083405 8972 leader_election.cc:343] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.083425 9101 ts_tablet_manager.cc:1434] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.019s user 0.003s sys 0.003s
I20260807 19:48:51.087419 9101 tablet_bootstrap.cc:492] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:51.093572 9101 tablet_bootstrap.cc:654] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.094019 9137 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.100682 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:51.101403 8906 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.107136 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.107898 8911 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:51.108242 8629 catalog_manager.cc:5697] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:51.113873 9388 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.114419 9388 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.116693 8972 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:51.118973 9413 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.119376 9413 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.119717 9413 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.118252 9388 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.125360 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.129454 8908 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:51.131611 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:51.132639 8906 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:51.128712 9137 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.135003 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:51.135727 9137 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.132517 9388 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.136392 9388 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.136754 9388 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.136200 9137 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:51.137872 9137 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.139204 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:51.141537 8740 leader_election.cc:343] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.142700 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:48:51.144497 8740 leader_election.cc:343] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.149297 9388 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.148686 9413 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.151489 9413 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.152138 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:51.152313 9388 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.152793 8906 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.153446 9101 tablet_bootstrap.cc:492] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:51.153887 9101 ts_tablet_manager.cc:1403] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.067s user 0.024s sys 0.012s
I20260807 19:48:51.154470 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:51.156100 8632 catalog_manager.cc:5697] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:51.157734 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:51.159080 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:51.159768 8908 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.161401 9099 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:51.162465 9099 ts_tablet_manager.cc:1403] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.110s user 0.015s sys 0.013s
I20260807 19:48:51.162395 9101 raft_consensus.cc:359] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.163193 9101 raft_consensus.cc:385] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.163483 9101 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.164166 9101 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.165383 9099 raft_consensus.cc:359] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.166077 9099 raft_consensus.cc:385] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.159181 9106 tablet_bootstrap.cc:654] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.166325 9099 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
W20260807 19:48:51.156116 8968 leader_election.cc:343] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:51.170759 8740 leader_election.cc:343] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.167069 9099 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.174619 9099 ts_tablet_manager.cc:1434] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260807 19:48:51.175766 9099 tablet_bootstrap.cc:492] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
W20260807 19:48:51.181083 8856 consensus_peers.cc:597] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 -> Peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Couldn't send request to peer a38f4d6088cf4c8cb510200ca0555801. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:51.170912 9101 ts_tablet_manager.cc:1434] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260807 19:48:51.183369 9101 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:51.188813 8906 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:51.192752 9106 tablet_bootstrap.cc:492] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:51.193260 9106 ts_tablet_manager.cc:1403] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.142s user 0.016s sys 0.014s
I20260807 19:48:51.195843 9106 raft_consensus.cc:359] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.196560 9106 raft_consensus.cc:385] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.196846 9106 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.197479 9106 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.199514 9106 ts_tablet_manager.cc:1434] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 19:48:51.190140 9101 tablet_bootstrap.cc:654] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.200291 9106 tablet_bootstrap.cc:492] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:51.205972 9405 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.207929 8908 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:51.206549 9099 tablet_bootstrap.cc:654] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.209841 8972 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:51.206489 9405 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.213160 9405 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.214785 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.215446 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.215648 9117 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.215780 9413 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.216331 9117 raft_consensus.cc:697] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:51.216408 9413 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:51.218120 9117 consensus_queue.cc:237] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.222194 9106 tablet_bootstrap.cc:654] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.225600 9413 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.230198 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.231048 8787 raft_consensus.cc:2393] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:51.232209 8849 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.233278 9405 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.242136 9101 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:51.242614 9101 ts_tablet_manager.cc:1403] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.008s
I20260807 19:48:51.242924 9405 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:51.245234 9106 tablet_bootstrap.cc:492] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:51.245743 9106 ts_tablet_manager.cc:1403] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.003s
I20260807 19:48:51.246018 9101 raft_consensus.cc:359] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.248775 9101 raft_consensus.cc:385] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.249069 9101 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.250267 9409 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.250665 9101 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.250792 9409 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.251271 8632 catalog_manager.cc:5697] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:51.252967 9409 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.253470 9101 ts_tablet_manager.cc:1434] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:48:51.255483 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.255740 9106 raft_consensus.cc:359] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.256126 8787 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:51.256420 9106 raft_consensus.cc:385] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.256872 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:51.258286 8972 leader_election.cc:343] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.259855 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:48:51.249719 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:51.260901 9409 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.261189 9409 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.261484 9409 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.262946 8629 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:51.256742 9106 raft_consensus.cc:740] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.267730 9276 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:51.267470 9106 consensus_queue.cc:260] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.270257 9106 ts_tablet_manager.cc:1434] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
I20260807 19:48:51.270700 9276 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.271163 9276 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.272001 9106 tablet_bootstrap.cc:492] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:51.273959 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.274717 8779 raft_consensus.cc:2393] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:51.278162 9276 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.269721 9409 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.281234 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:51.281816 8779 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.283013 9099 tablet_bootstrap.cc:492] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:51.283474 9099 ts_tablet_manager.cc:1403] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.108s user 0.011s sys 0.001s
I20260807 19:48:51.284062 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:48:51.285432 8972 leader_election.cc:343] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.285966 9409 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.286649 9099 raft_consensus.cc:359] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.287384 9099 raft_consensus.cc:385] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.287644 9099 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.281716 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.288506 9099 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.290143 8856 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.290652 9099 ts_tablet_manager.cc:1434] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:48:51.291040 9276 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.293768 9101 tablet_bootstrap.cc:492] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:51.293977 9106 tablet_bootstrap.cc:654] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.291610 9099 tablet_bootstrap.cc:492] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:51.299575 8779 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:51.306870 9276 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:51.308184 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:51.309722 9409 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.310858 9409 raft_consensus.cc:697] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:51.307180 9101 tablet_bootstrap.cc:654] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.303094 9099 tablet_bootstrap.cc:654] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.314332 9106 tablet_bootstrap.cc:492] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:51.313093 9413 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.321086 9413 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.320480 9106 ts_tablet_manager.cc:1403] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.049s user 0.016s sys 0.012s
I20260807 19:48:51.326815 9106 raft_consensus.cc:359] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.327510 9106 raft_consensus.cc:385] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.327775 9106 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.328457 9106 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.331135 9101 tablet_bootstrap.cc:492] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:51.331259 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.331964 8908 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:51.333320 8972 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:51.334965 9145 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.335356 9145 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.335431 9106 ts_tablet_manager.cc:1434] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260807 19:48:51.336603 9106 tablet_bootstrap.cc:492] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:51.336866 9145 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.336830 9101 ts_tablet_manager.cc:1403] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.003s
I20260807 19:48:51.337798 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:48:51.339866 8968 leader_election.cc:343] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.312798 9409 consensus_queue.cc:237] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.341641 9101 raft_consensus.cc:359] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.342433 9101 raft_consensus.cc:385] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.343112 9117 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.343633 9117 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.336959 9413 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.342733 9101 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.346715 9117 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.346899 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.347386 9101 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.348588 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.349689 9101 ts_tablet_manager.cc:1434] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260807 19:48:51.347909 8908 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:51.350925 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:48:51.351999 8740 leader_election.cc:343] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.353881 9117 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.354337 9117 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.354831 9117 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.355013 9106 tablet_bootstrap.cc:654] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.358506 9101 tablet_bootstrap.cc:492] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:51.364616 9145 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.367488 9413 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.368682 9413 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.372641 9413 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.373134 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.373880 8787 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:51.374894 9099 tablet_bootstrap.cc:492] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:51.375422 9099 ts_tablet_manager.cc:1403] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.084s user 0.015s sys 0.003s
I20260807 19:48:51.375294 8968 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:51.377858 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.381491 9099 raft_consensus.cc:359] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.383356 9099 raft_consensus.cc:385] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260807 19:48:51.385232 8972 leader_election.cc:343] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.384485 9099 raft_consensus.cc:740] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.386716 9492 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.388757 9099 consensus_queue.cc:260] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.393447 9099 ts_tablet_manager.cc:1434] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.018s user 0.007s sys 0.001s
I20260807 19:48:51.388248 9492 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.387427 9413 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.408128 9492 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.409169 9099 tablet_bootstrap.cc:492] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:51.411258 9145 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.413463 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.414217 8780 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:51.415815 9106 tablet_bootstrap.cc:492] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:51.416396 9106 ts_tablet_manager.cc:1403] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.080s user 0.022s sys 0.016s
I20260807 19:48:51.416517 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:51.417898 8908 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.417557 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:51.416965 9117 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.400225 9413 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.421885 9192 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.423133 8632 catalog_manager.cc:5697] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:51.423756 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:51.423872 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.424177 9101 tablet_bootstrap.cc:654] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.428817 9192 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.429273 9192 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.430331 8780 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:51.437403 8856 leader_election.cc:343] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.437752 9351 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:51.445744 9117 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.443365 9106 raft_consensus.cc:359] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.433936 9413 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.448592 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:51.448705 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:51.452178 8906 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.448180 9106 raft_consensus.cc:385] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.453887 9106 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
W20260807 19:48:51.454274 8740 leader_election.cc:343] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.454700 9106 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.448441 9192 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.458597 9106 ts_tablet_manager.cc:1434] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.041s user 0.006s sys 0.000s
I20260807 19:48:51.460055 9106 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:51.461939 8780 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:51.463313 8968 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:51.464147 9351 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.464701 9351 raft_consensus.cc:697] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:51.465572 9351 consensus_queue.cc:237] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.466619 9101 tablet_bootstrap.cc:492] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:51.467103 9101 ts_tablet_manager.cc:1403] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.109s user 0.013s sys 0.002s
I20260807 19:48:51.471403 9192 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.471588 9099 tablet_bootstrap.cc:654] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.452127 8908 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:51.474779 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:51.475612 8780 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.477982 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:51.482151 9117 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.487262 8906 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:51.509163 9101 raft_consensus.cc:359] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:51.509720 8856 leader_election.cc:343] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.509917 9101 raft_consensus.cc:385] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.489224 9117 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.511217 9101 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.513770 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.518760 9137 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.519243 9137 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:51.520133 9101 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.519984 9137 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.541414 9101 ts_tablet_manager.cc:1434] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.074s user 0.019s sys 0.018s
I20260807 19:48:51.542642 9101 tablet_bootstrap.cc:492] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:51.544638 9099 tablet_bootstrap.cc:492] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:51.545224 9099 ts_tablet_manager.cc:1403] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.136s user 0.010s sys 0.013s
I20260807 19:48:51.550236 9106 tablet_bootstrap.cc:654] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:51.557093 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.543915 9117 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.557924 8906 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:51.562533 8780 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:51.531539 9351 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260807 19:48:51.576941 9099 raft_consensus.cc:359] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.577841 9099 raft_consensus.cc:385] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.577672 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.578354 9099 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.579125 9099 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.569804 9276 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.553901 9413 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.587355 9192 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.588128 9492 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.637588 8628 catalog_manager.cc:5697] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:51.587123 9276 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.608678 9413 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.632298 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:51.632133 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.656546 9137 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:51.659282 9099 ts_tablet_manager.cc:1434] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.114s user 0.009s sys 0.009s
I20260807 19:48:51.660436 9099 tablet_bootstrap.cc:492] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:51.640918 9192 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.646973 9351 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.664563 9517 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260807 19:48:51.674624 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:51.675320 8789 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.684445 9351 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.686580 9351 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.687196 9192 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.693463 9276 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.696187 9101 tablet_bootstrap.cc:654] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:51.714228 8972 leader_election.cc:343] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:51.699112 8789 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:51.725693 9117 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.727365 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.720938 9413 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260807 19:48:51.710965 9492 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:48:51.742825 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 68433.6us
I20260807 19:48:51.742699 9117 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.752667 9117 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:51.749429 9517 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:51.758924 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.759778 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.760473 8788 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:51.760944 9492 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.763731 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.764145 8783 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.764730 8780 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:51.770605 8968 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:51.785005 9106 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:51.785647 9106 ts_tablet_manager.cc:1403] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.326s user 0.018s sys 0.036s
I20260807 19:48:51.780689 9413 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260807 19:48:51.758690 9517 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.787317 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.776649 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.791749 9019 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.798198 9517 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:51.799412 9106 raft_consensus.cc:359] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.799955 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.800675 8913 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:51.801783 9020 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:51.802152 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:51.786329 9511 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:51.802871 9511 raft_consensus.cc:697] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:51.803145 9523 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.803529 9523 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.803881 9523 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.778571 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.800176 9106 raft_consensus.cc:385] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.805508 9106 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.806264 9106 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.811301 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:51.819361 9106 ts_tablet_manager.cc:1434] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260807 19:48:51.824779 9101 tablet_bootstrap.cc:492] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:51.825376 9101 ts_tablet_manager.cc:1403] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.283s user 0.018s sys 0.004s
I20260807 19:48:51.833426 8632 catalog_manager.cc:5697] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:51.847527 9137 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:51.853057 9020 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.832027 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:51.826752 9520 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:51.856293 9106 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:51.861654 9145 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:48:51.867431 8740 consensus_peers.cc:597] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Couldn't send request to peer a38f4d6088cf4c8cb510200ca0555801. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:51.870090 9145 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:51.873155 8968 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:51.857611 9520 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:51.803735 9511 consensus_queue.cc:237] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.888300 9101 raft_consensus.cc:359] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.894775 9101 raft_consensus.cc:385] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:51.895040 9101 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:51.883435 9520 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.896275 9101 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.903604 9101 ts_tablet_manager.cc:1434] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.078s user 0.009s sys 0.000s
I20260807 19:48:51.903826 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:51.904651 9101 tablet_bootstrap.cc:492] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:51.906576 9099 tablet_bootstrap.cc:654] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:51.907603 8740 consensus_peers.cc:597] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Couldn't send request to peer a38f4d6088cf4c8cb510200ca0555801. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:51.917269 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:51.918088 8787 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:51.921490 8856 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:51.924317 9276 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:51.942982 8628 catalog_manager.cc:5697] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:51.934669 9482 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260807 19:48:51.952776 9523 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:51.955302 9023 raft_consensus.cc:2393] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:51.962898 9351 consensus_queue.cc:1048] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260807 19:48:51.966871 9522 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260807 19:48:51.974902 9276 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:51.994266 9409 consensus_queue.cc:1048] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:52.003576 9106 tablet_bootstrap.cc:654] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.014066 9409 consensus_queue.cc:1048] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.044832 9523 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:52.046733 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:52.067106 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:52.067790 8790 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:52.090137 9409 consensus_queue.cc:1048] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.106844 9101 tablet_bootstrap.cc:654] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.094383 8790 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:52.110378 9145 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.119199 9528 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.126398 9099 tablet_bootstrap.cc:492] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:52.127141 9099 ts_tablet_manager.cc:1403] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.467s user 0.013s sys 0.009s
I20260807 19:48:52.130767 8968 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:52.140247 9145 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:52.140857 9145 raft_consensus.cc:697] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:52.141710 9145 consensus_queue.cc:237] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:52.150754 9548 log.cc:927] Time spent T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.121s user 0.004s sys 0.000s
I20260807 19:48:52.152055 9482 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:52.139424 9520 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.158344 8632 catalog_manager.cc:5697] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:52.189154 8903 raft_consensus.cc:2393] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:52.190337 9099 raft_consensus.cc:359] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.196238 8740 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.196569 9099 raft_consensus.cc:385] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.204655 9099 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.211599 9099 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.218719 9540 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.231618 9520 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:48:52.227401 9551 log.cc:927] Time spent T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.069s user 0.003s sys 0.008s
I20260807 19:48:52.239114 9099 ts_tablet_manager.cc:1434] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.112s user 0.006s sys 0.003s
I20260807 19:48:52.239998 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:52.240334 9099 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:52.240767 8789 raft_consensus.cc:2393] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:52.308981 9106 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:52.309546 9106 ts_tablet_manager.cc:1403] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.453s user 0.047s sys 0.042s
I20260807 19:48:52.311002 9099 tablet_bootstrap.cc:654] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.309618 9101 tablet_bootstrap.cc:492] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:52.312330 9106 raft_consensus.cc:359] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.313129 9106 raft_consensus.cc:385] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.313520 9106 raft_consensus.cc:740] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.313187 9101 ts_tablet_manager.cc:1403] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.409s user 0.028s sys 0.022s
I20260807 19:48:52.314227 9106 consensus_queue.cc:260] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.318360 9106 ts_tablet_manager.cc:1434] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260807 19:48:52.319487 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:52.320204 9106 tablet_bootstrap.cc:492] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:52.327033 9106 tablet_bootstrap.cc:654] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.329929 9099 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:52.387960 9099 ts_tablet_manager.cc:1403] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.148s user 0.013s sys 0.000s
I20260807 19:48:52.394120 9099 raft_consensus.cc:359] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.395888 9099 raft_consensus.cc:385] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.397008 9388 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.404657 9388 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.408735 9388 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.396315 9099 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.424587 9200 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.425545 9200 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.420336 9101 raft_consensus.cc:359] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.427934 9101 raft_consensus.cc:385] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.429512 9106 tablet_bootstrap.cc:492] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:52.430387 9200 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.431182 9099 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.432269 9101 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.433322 9101 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.436061 9099 ts_tablet_manager.cc:1434] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.047s user 0.003s sys 0.008s
I20260807 19:48:52.438113 9099 tablet_bootstrap.cc:492] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:52.438872 8856 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.430545 9106 ts_tablet_manager.cc:1403] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.111s user 0.009s sys 0.006s
I20260807 19:48:52.440630 9573 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.449278 9521 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.449980 9521 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.460240 9106 raft_consensus.cc:359] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.456789 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:52.463452 9106 raft_consensus.cc:385] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.464170 9106 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.466009 9521 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.460433 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:52.466038 9106 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.472478 9106 ts_tablet_manager.cc:1434] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.031s user 0.007s sys 0.003s
I20260807 19:48:52.472711 9101 ts_tablet_manager.cc:1434] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.158s user 0.014s sys 0.007s
I20260807 19:48:52.474058 9106 tablet_bootstrap.cc:492] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:52.474493 9101 tablet_bootstrap.cc:492] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
W20260807 19:48:52.482193 8856 consensus_peers.cc:597] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 -> Peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Couldn't send request to peer a38f4d6088cf4c8cb510200ca0555801. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260807 19:48:52.490949 8856 leader_election.cc:343] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.496698 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:52.501700 9099 tablet_bootstrap.cc:654] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.496765 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:52.509958 8740 leader_election.cc:343] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:52.515403 8740 leader_election.cc:343] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.516860 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.517928 9137 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.532404 9027 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:52.535022 8903 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:52.538228 9106 tablet_bootstrap.cc:654] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.545619 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:52.545641 9145 consensus_queue.cc:1048] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.549237 9200 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:52.550047 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:52.553473 8783 raft_consensus.cc:2393] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:52.569403 9101 tablet_bootstrap.cc:654] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:52.580268 8849 leader_election.cc:343] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.580865 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.591238 9099 tablet_bootstrap.cc:492] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:52.591933 9099 ts_tablet_manager.cc:1403] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.154s user 0.024s sys 0.015s
I20260807 19:48:52.595965 8849 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.591035 8632 catalog_manager.cc:5697] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:52.606504 9580 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.600977 9200 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.616748 9517 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.617425 9517 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.639240 9517 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:52.647621 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:52.648710 9099 raft_consensus.cc:359] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.649533 9099 raft_consensus.cc:385] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.649761 9099 raft_consensus.cc:740] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.653720 9020 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:52.663666 9409 consensus_queue.cc:1048] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.674803 9517 consensus_queue.cc:1048] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.693889 9099 consensus_queue.cc:260] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.703135 9106 tablet_bootstrap.cc:492] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:52.703778 9106 ts_tablet_manager.cc:1403] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.230s user 0.013s sys 0.009s
I20260807 19:48:52.707616 9099 ts_tablet_manager.cc:1434] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.115s user 0.011s sys 0.007s
I20260807 19:48:52.712098 9106 raft_consensus.cc:359] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.713091 9106 raft_consensus.cc:385] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.713383 9106 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.714155 9106 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.715204 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:52.712325 9099 tablet_bootstrap.cc:492] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:52.773113 9106 ts_tablet_manager.cc:1434] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.069s user 0.009s sys 0.001s
I20260807 19:48:52.774401 9106 tablet_bootstrap.cc:492] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:52.783468 8972 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:52.784225 9099 tablet_bootstrap.cc:654] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:52.784315 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:52.785442 9101 tablet_bootstrap.cc:492] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:52.785938 9101 ts_tablet_manager.cc:1403] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.312s user 0.015s sys 0.018s
I20260807 19:48:52.792701 9413 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.804991 9516 consensus_queue.cc:1048] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:52.808835 9101 raft_consensus.cc:359] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.811913 9106 tablet_bootstrap.cc:654] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.811980 9101 raft_consensus.cc:385] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.812577 9101 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.813344 9101 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.815963 9101 ts_tablet_manager.cc:1434] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.030s user 0.007s sys 0.000s
I20260807 19:48:52.820256 9099 tablet_bootstrap.cc:492] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:52.820793 9099 ts_tablet_manager.cc:1403] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.109s user 0.013s sys 0.001s
I20260807 19:48:52.821486 9579 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.821972 9579 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.823531 9099 raft_consensus.cc:359] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.824205 9099 raft_consensus.cc:385] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.827618 9099 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.828307 9099 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.830886 9099 ts_tablet_manager.cc:1434] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.010s user 0.003s sys 0.003s
I20260807 19:48:52.830737 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:52.832158 9101 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
W20260807 19:48:52.834776 8740 leader_election.cc:343] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.836409 9137 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.836848 9137 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.838698 9137 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.839597 9101 tablet_bootstrap.cc:654] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.824120 9579 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.843904 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:52.846683 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:52.847445 8740 leader_election.cc:343] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.848816 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:52.850612 8740 leader_election.cc:343] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.851049 8740 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.852239 9388 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.853189 9099 tablet_bootstrap.cc:492] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:52.881223 9106 tablet_bootstrap.cc:492] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:52.881848 9106 ts_tablet_manager.cc:1403] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.108s user 0.008s sys 0.003s
W20260807 19:48:52.887205 8740 leader_election.cc:343] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.887612 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.889055 9388 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:52.894559 9106 raft_consensus.cc:359] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.895321 9106 raft_consensus.cc:385] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.895591 9106 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.896464 9106 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.901885 9106 ts_tablet_manager.cc:1434] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260807 19:48:52.904318 9106 tablet_bootstrap.cc:492] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:52.902503 9099 tablet_bootstrap.cc:654] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.913048 9276 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:52.913645 9276 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.915865 9276 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.917083 9099 tablet_bootstrap.cc:492] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:52.917654 9099 ts_tablet_manager.cc:1403] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.066s user 0.015s sys 0.001s
I20260807 19:48:52.917812 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:52.919406 8856 leader_election.cc:343] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.920938 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:52.921759 8787 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:52.926723 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.927970 9276 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:52.927697 9099 raft_consensus.cc:359] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.928416 9276 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:52.928817 9276 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:52.930145 9099 raft_consensus.cc:385] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.930522 9099 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.931121 9099 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.937877 9276 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.939656 9099 ts_tablet_manager.cc:1434] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.021s user 0.005s sys 0.002s
I20260807 19:48:52.944010 9099 tablet_bootstrap.cc:492] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:52.944785 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:52.945196 9106 tablet_bootstrap.cc:654] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.945393 8787 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:52.945868 9101 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:52.946338 9101 ts_tablet_manager.cc:1403] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.114s user 0.014s sys 0.007s
I20260807 19:48:52.949211 9101 raft_consensus.cc:359] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.949937 9101 raft_consensus.cc:385] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.950214 9101 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.950871 9101 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.953120 9101 ts_tablet_manager.cc:1434] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260807 19:48:52.944070 9276 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:52.953567 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:52.954856 9101 tablet_bootstrap.cc:492] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:52.959142 9099 tablet_bootstrap.cc:654] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:52.961668 9101 tablet_bootstrap.cc:654] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:52.971580 8856 leader_election.cc:343] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:52.968439 9099 tablet_bootstrap.cc:492] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:52.974551 9099 ts_tablet_manager.cc:1403] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.007s
I20260807 19:48:52.975953 8787 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:52.973592 9101 tablet_bootstrap.cc:492] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:52.976960 9101 ts_tablet_manager.cc:1403] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.008s
I20260807 19:48:52.977686 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:52.978546 9276 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:52.979118 9276 raft_consensus.cc:697] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:52.980010 9276 consensus_queue.cc:237] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.980245 9099 raft_consensus.cc:359] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.980921 9099 raft_consensus.cc:385] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.981182 9099 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.984460 9101 raft_consensus.cc:359] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.985199 9101 raft_consensus.cc:385] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.985466 9101 raft_consensus.cc:740] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.986140 9101 consensus_queue.cc:260] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.988270 9101 ts_tablet_manager.cc:1434] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260807 19:48:52.989439 9101 tablet_bootstrap.cc:492] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:52.992026 9106 tablet_bootstrap.cc:492] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:52.992487 9106 ts_tablet_manager.cc:1403] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.088s user 0.009s sys 0.014s
I20260807 19:48:52.992504 9099 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.994258 9099 ts_tablet_manager.cc:1434] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.019s user 0.005s sys 0.000s
I20260807 19:48:52.995236 9099 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:52.995486 9106 raft_consensus.cc:359] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.996186 9106 raft_consensus.cc:385] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:52.996496 9106 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:52.997195 9106 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:52.999544 9106 ts_tablet_manager.cc:1434] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:48:53.001286 9106 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.018009 9101 tablet_bootstrap.cc:654] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.021132 9106 tablet_bootstrap.cc:654] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.021940 9099 tablet_bootstrap.cc:654] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.043623 9482 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.044291 9482 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.046679 9482 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.052838 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.053678 8908 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:53.054792 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:53.055576 9022 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:53.062881 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:53.064258 9579 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.064724 9579 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.065065 9579 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.075910 9388 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.076538 9388 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.078563 9388 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.082809 9573 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.083357 9573 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.085517 9573 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.085645 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:53.087563 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:53.088817 8856 leader_election.cc:343] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
W20260807 19:48:53.090476 8740 leader_election.cc:343] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.090497 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.091394 8789 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:53.092897 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.093673 8911 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:53.110919 9409 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.111531 9409 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.114288 9409 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.117483 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.118243 8789 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:53.120069 8968 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:53.121345 9523 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.121767 9523 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.122102 9523 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.130990 9523 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.132143 9101 tablet_bootstrap.cc:492] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.132164 9106 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.132862 9101 ts_tablet_manager.cc:1403] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.144s user 0.042s sys 0.059s
I20260807 19:48:53.132947 9106 ts_tablet_manager.cc:1403] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.132s user 0.047s sys 0.054s
I20260807 19:48:53.134305 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:53.134011 8628 catalog_manager.cc:5697] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:53.135931 9101 raft_consensus.cc:359] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.136700 9101 raft_consensus.cc:385] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.136246 9106 raft_consensus.cc:359] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.136996 9101 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.137161 9106 raft_consensus.cc:385] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.137552 9106 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.138386 9106 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.138410 9101 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.132794 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.137449 8849 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.140977 9276 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.141032 9101 ts_tablet_manager.cc:1434] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:48:53.141103 9106 ts_tablet_manager.cc:1434] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:48:53.141492 9276 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.141886 9276 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.142463 9106 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.142338 9101 tablet_bootstrap.cc:492] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:53.142117 9099 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:53.143388 9099 ts_tablet_manager.cc:1403] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.148s user 0.043s sys 0.044s
I20260807 19:48:53.149333 9101 tablet_bootstrap.cc:654] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.149573 9106 tablet_bootstrap.cc:654] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:53.149924 8972 leader_election.cc:343] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.150121 9523 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.151358 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.149147 9099 raft_consensus.cc:359] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:53.152980 8972 leader_election.cc:343] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.153803 9099 raft_consensus.cc:385] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.157030 9099 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.160985 9101 tablet_bootstrap.cc:492] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.162364 9106 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.163029 9106 ts_tablet_manager.cc:1403] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.008s
I20260807 19:48:53.162369 9099 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.162808 9101 ts_tablet_manager.cc:1403] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.009s
I20260807 19:48:53.153966 9276 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.166105 9106 raft_consensus.cc:359] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.166534 9276 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.166817 9106 raft_consensus.cc:385] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.167258 9106 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.167619 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:53.168196 9101 raft_consensus.cc:359] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.168077 9106 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:53.168916 8856 leader_election.cc:343] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.169085 8789 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.170976 9106 ts_tablet_manager.cc:1434] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:48:53.172163 9106 tablet_bootstrap.cc:492] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.172035 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.168932 9101 raft_consensus.cc:385] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.173086 9101 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.173785 9137 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.174153 9137 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.174448 9137 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.181406 9099 ts_tablet_manager.cc:1434] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260807 19:48:53.178510 9580 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.182578 9106 tablet_bootstrap.cc:654] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.182816 9580 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.183633 9101 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.186013 9580 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.187107 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:53.188591 8856 leader_election.cc:343] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.189327 9099 tablet_bootstrap.cc:492] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:53.181965 9137 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.192732 9137 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.193480 9579 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.196411 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:53.199635 8787 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.202852 9101 ts_tablet_manager.cc:1434] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.038s user 0.005s sys 0.001s
I20260807 19:48:53.205044 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.205668 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.206260 8911 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.209282 8787 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:53.210819 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:53.203980 9101 tablet_bootstrap.cc:492] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:53.217192 9021 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.211362 9579 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.181756 8789 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:53.226578 8968 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:53.227568 9409 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.228093 9409 raft_consensus.cc:697] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:53.228911 9101 tablet_bootstrap.cc:654] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.229235 9409 consensus_queue.cc:237] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.231145 9099 tablet_bootstrap.cc:654] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.239468 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.240281 8903 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
W20260807 19:48:53.241277 8849 leader_election.cc:343] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.241743 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.242307 9409 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.245239 9409 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.246431 9145 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.246955 9145 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.247690 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:53.248315 9022 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.249173 9145 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.249953 9021 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:53.250873 9580 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:53.251588 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.256328 9101 tablet_bootstrap.cc:492] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.256847 9101 ts_tablet_manager.cc:1403] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.053s user 0.012s sys 0.004s
I20260807 19:48:53.259225 9022 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:53.259481 9101 raft_consensus.cc:359] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.260221 9101 raft_consensus.cc:385] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.260506 9101 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.261149 9101 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.262606 9101 ts_tablet_manager.cc:1434] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:48:53.263684 9101 tablet_bootstrap.cc:492] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:53.264022 8903 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:53.269375 9409 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.271018 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.273304 8790 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:53.273550 9101 tablet_bootstrap.cc:654] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.274576 8968 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:53.275656 9409 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.276031 9409 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.276626 9409 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.288712 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.289928 8849 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.290943 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.292428 9580 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.293079 9580 raft_consensus.cc:697] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:53.294340 9580 consensus_queue.cc:237] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.295207 9099 tablet_bootstrap.cc:492] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:53.295852 9099 ts_tablet_manager.cc:1403] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.109s user 0.013s sys 0.000s
I20260807 19:48:53.300225 9106 tablet_bootstrap.cc:492] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.300945 9106 ts_tablet_manager.cc:1403] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.129s user 0.028s sys 0.042s
I20260807 19:48:53.301066 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:53.301993 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:53.302136 9579 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.302744 9137 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.303210 9137 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:53.304064 9137 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.271243 8911 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:53.309146 9579 raft_consensus.cc:697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:53.310166 9579 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.310537 9409 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.312251 9099 raft_consensus.cc:359] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.313020 9099 raft_consensus.cc:385] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.313293 9099 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.314610 9106 raft_consensus.cc:359] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.315394 9106 raft_consensus.cc:385] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.315711 9106 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.316659 9106 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.318953 9106 ts_tablet_manager.cc:1434] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260807 19:48:53.319056 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:53.326315 8790 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.327589 9106 tablet_bootstrap.cc:492] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.320938 9409 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.320788 9099 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:53.339148 8972 leader_election.cc:343] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.325151 8628 catalog_manager.cc:5697] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:53.342314 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:53.343772 9145 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:53.337622 8790 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:53.347987 9099 ts_tablet_manager.cc:1434] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.052s user 0.008s sys 0.000s
I20260807 19:48:53.349130 9099 tablet_bootstrap.cc:492] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:53.349737 8968 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:53.351186 9523 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.351737 9523 raft_consensus.cc:697] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:53.354725 9523 consensus_queue.cc:237] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.357486 8632 catalog_manager.cc:5697] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:53.365237 9579 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.365826 9579 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.370342 9099 tablet_bootstrap.cc:654] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.359119 8632 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:53.375029 9579 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.378170 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:53.379522 8740 leader_election.cc:343] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.380394 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.383078 9101 tablet_bootstrap.cc:492] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.383615 9101 ts_tablet_manager.cc:1403] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.120s user 0.035s sys 0.041s
I20260807 19:48:53.379336 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.388862 9106 tablet_bootstrap.cc:654] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.394675 9409 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.394456 9101 raft_consensus.cc:359] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.395195 9409 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:53.395602 8740 leader_election.cc:343] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.396056 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.395205 9101 raft_consensus.cc:385] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.397223 9409 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.396497 8629 catalog_manager.cc:5697] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:53.397436 9101 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.397619 9579 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:53.398172 9101 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.400238 9101 ts_tablet_manager.cc:1434] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20260807 19:48:53.401325 9101 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:53.401499 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:48:53.402815 8968 leader_election.cc:343] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.406638 9106 tablet_bootstrap.cc:492] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.407069 9106 ts_tablet_manager.cc:1403] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.080s user 0.006s sys 0.005s
I20260807 19:48:53.408676 8628 catalog_manager.cc:5697] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:48:53.406664 8972 leader_election.cc:343] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.409925 9106 raft_consensus.cc:359] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.410606 9106 raft_consensus.cc:385] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.410899 9106 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.411577 9106 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.416486 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.417275 8903 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:53.421011 9106 ts_tablet_manager.cc:1434] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260807 19:48:53.422219 9106 tablet_bootstrap.cc:492] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.425746 9101 tablet_bootstrap.cc:654] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.428485 9106 tablet_bootstrap.cc:654] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.436710 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:53.442942 9101 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.443490 9101 ts_tablet_manager.cc:1403] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.008s
I20260807 19:48:53.443583 9413 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.444036 9413 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.444386 9413 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.445708 9106 tablet_bootstrap.cc:492] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.446364 9106 ts_tablet_manager.cc:1403] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.005s
I20260807 19:48:53.449038 9101 raft_consensus.cc:359] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.449223 9106 raft_consensus.cc:359] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.449729 9101 raft_consensus.cc:385] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.449856 9106 raft_consensus.cc:385] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.450044 9101 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.450138 9106 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.450996 9106 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.451263 9101 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.452751 9573 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.453176 9106 ts_tablet_manager.cc:1434] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:48:53.453286 9101 ts_tablet_manager.cc:1434] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:48:53.454289 9106 tablet_bootstrap.cc:492] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.453289 9573 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.454337 9101 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:53.457858 9021 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.462565 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:53.463438 9573 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:48:53.464632 8856 leader_election.cc:343] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.465583 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.466760 9106 tablet_bootstrap.cc:654] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:53.467027 8849 leader_election.cc:343] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.453579 9413 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.467593 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.468567 9580 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:53.473379 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:48:53.477289 8968 leader_election.cc:343] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.477634 9101 tablet_bootstrap.cc:654] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.485299 9106 tablet_bootstrap.cc:492] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.485790 9106 ts_tablet_manager.cc:1403] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.004s
I20260807 19:48:53.486883 9580 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.487391 9580 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.489538 9580 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.488334 9106 raft_consensus.cc:359] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.493808 9106 raft_consensus.cc:385] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.494073 9106 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.497562 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.498181 8789 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:53.500443 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:53.501948 8856 leader_election.cc:343] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.503716 8849 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.504194 9409 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.504798 9409 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.505502 9580 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.505929 9580 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.506278 9580 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.508131 9409 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.505437 9413 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.513949 9615 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.515125 9615 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.514274 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.515576 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.516022 8903 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.519552 9101 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.520092 9101 ts_tablet_manager.cc:1403] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.066s user 0.006s sys 0.006s
I20260807 19:48:53.521802 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.522535 8911 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:53.523720 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:53.524741 9615 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.526072 9615 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.526847 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.527724 9099 tablet_bootstrap.cc:492] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:53.527899 8789 raft_consensus.cc:2393] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:53.528263 9106 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.529987 9106 ts_tablet_manager.cc:1434] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260807 19:48:53.530156 8968 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:53.530432 9101 raft_consensus.cc:359] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.531141 9101 raft_consensus.cc:385] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.531419 9101 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.531766 9099 ts_tablet_manager.cc:1403] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.183s user 0.025s sys 0.023s
I20260807 19:48:53.531985 9145 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.532750 8903 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:53.535501 9101 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.538272 9573 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:53.535996 9615 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.540658 9615 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.542769 9145 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:53.540444 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:53.544658 8740 leader_election.cc:343] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.546195 9099 raft_consensus.cc:359] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.546947 9099 raft_consensus.cc:385] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.547274 9099 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.548008 9099 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.550182 9099 ts_tablet_manager.cc:1434] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260807 19:48:53.551046 9099 tablet_bootstrap.cc:492] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:53.533179 9106 tablet_bootstrap.cc:492] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:53.556893 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:53.557770 9145 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.558171 9145 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:53.558758 9145 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.560869 9580 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.570895 9573 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:53.572837 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:53.573478 8789 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.578256 9106 tablet_bootstrap.cc:654] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.580011 9099 tablet_bootstrap.cc:654] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.590233 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:53.580039 9615 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:53.591696 8856 leader_election.cc:343] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.581372 8789 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:53.583539 9580 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.593055 8849 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.593161 9615 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.597133 9101 ts_tablet_manager.cc:1434] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.077s user 0.014s sys 0.020s
I20260807 19:48:53.599040 9106 tablet_bootstrap.cc:492] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:53.599509 9106 ts_tablet_manager.cc:1403] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.069s user 0.010s sys 0.002s
I20260807 19:48:53.600813 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:53.601143 9573 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.598237 9101 tablet_bootstrap.cc:492] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
W20260807 19:48:53.603092 8740 leader_election.cc:343] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.604316 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.605010 8903 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.615187 9101 tablet_bootstrap.cc:654] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.629072 9145 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:53.619419 8628 catalog_manager.cc:5697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:53.647511 8903 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:53.655182 9106 raft_consensus.cc:359] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.656006 9106 raft_consensus.cc:385] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.656319 9099 tablet_bootstrap.cc:492] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:53.656967 9099 ts_tablet_manager.cc:1403] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.106s user 0.035s sys 0.024s
I20260807 19:48:53.659911 9099 raft_consensus.cc:359] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.664798 9099 raft_consensus.cc:385] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.665156 9099 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.672853 9099 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.675143 9106 raft_consensus.cc:740] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.676365 9106 consensus_queue.cc:260] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.678704 9106 ts_tablet_manager.cc:1434] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.079s user 0.007s sys 0.000s
I20260807 19:48:53.683602 9145 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.687474 9145 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.696136 9145 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.696914 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.699549 9099 ts_tablet_manager.cc:1434] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.042s user 0.006s sys 0.000s
I20260807 19:48:53.701041 9660 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.701568 9660 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.703907 9660 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.704865 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.705550 8787 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:53.706724 8968 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:53.707048 9615 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.707777 9409 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.708107 9409 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.708446 9409 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.708662 9615 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:53.711040 9662 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:53.711380 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:53.711143 9615 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.711048 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.714520 8787 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:53.715337 9660 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:53.716089 9660 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.718247 9660 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:53.697896 9573 raft_consensus.cc:697] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:53.723330 9106 tablet_bootstrap.cc:492] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
W20260807 19:48:53.724922 8972 leader_election.cc:343] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.730410 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:53.731338 8783 raft_consensus.cc:2393] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:48:53.726312 9099 tablet_bootstrap.cc:492] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:53.728449 8849 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:53.729457 9409 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.752333 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:53.753024 9021 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:53.754956 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:53.759274 8856 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:53.760571 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:53.767346 9101 tablet_bootstrap.cc:492] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:53.770110 9137 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:53.770643 9101 ts_tablet_manager.cc:1403] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.173s user 0.029s sys 0.031s
W20260807 19:48:53.768790 8972 leader_election.cc:343] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:53.748565 9659 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:53.772408 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:53.774679 8789 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.729457 9573 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.771883 9580 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.774289 9659 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:53.789183 9659 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:53.801112 8789 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:53.810842 8968 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:48:53.813148 9409 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:53.815857 9409 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:53.816601 9409 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:53.816895 9101 raft_consensus.cc:359] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.817512 9101 raft_consensus.cc:385] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:53.817906 9101 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:53.817574 9409 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.830587 9101 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:53.935738 9101 ts_tablet_manager.cc:1434] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.165s user 0.015s sys 0.010s
I20260807 19:48:53.986018 9106 tablet_bootstrap.cc:654] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:53.941169 9580 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:54.007154 9101 tablet_bootstrap.cc:492] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:54.037356 9099 tablet_bootstrap.cc:654] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
W20260807 19:48:54.040997 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:54.012400 9659 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.043311 9659 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:54.090725 8911 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:54.091269 9517 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:54.092303 9517 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:54.050074 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 177141us
I20260807 19:48:54.098161 9482 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:54.098723 9482 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.100857 9482 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:54.113510 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:54.114784 8787 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:54.129038 8628 catalog_manager.cc:5697] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:54.142227 9106 tablet_bootstrap.cc:492] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:54.144333 9106 ts_tablet_manager.cc:1403] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.421s user 0.011s sys 0.018s
I20260807 19:48:54.149657 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:54.155855 9099 tablet_bootstrap.cc:492] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:54.156528 9099 ts_tablet_manager.cc:1403] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.430s user 0.010s sys 0.012s
I20260807 19:48:54.150399 9018 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:54.155135 9017 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:54.177419 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:54.170622 8787 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:54.182160 9099 raft_consensus.cc:359] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.182881 9099 raft_consensus.cc:385] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:54.183166 9099 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:54.186607 9099 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.189692 9517 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:54.192683 9099 ts_tablet_manager.cc:1434] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.036s user 0.005s sys 0.002s
I20260807 19:48:54.194106 9099 tablet_bootstrap.cc:492] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
W20260807 19:48:54.199728 8740 leader_election.cc:343] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:54.201903 8629 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:54.207136 9665 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:54.207595 9579 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:54.207705 9665 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.211104 9106 raft_consensus.cc:359] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.211859 9106 raft_consensus.cc:385] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:54.212116 9106 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:54.212685 9101 tablet_bootstrap.cc:654] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:54.212502 9137 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:54.215246 9615 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:54.215643 9106 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.218415 9106 ts_tablet_manager.cc:1434] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.074s user 0.008s sys 0.000s
I20260807 19:48:54.219561 9106 tablet_bootstrap.cc:492] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:54.221928 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:54.225548 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:54.220115 9018 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:54.230352 9665 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:54.232148 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:54.238724 8856 consensus_peers.cc:597] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 -> Peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Couldn't send request to peer a38f4d6088cf4c8cb510200ca0555801. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:54.238058 9659 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 19:48:54.239354 8856 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:54.241302 9659 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:54.241796 9659 raft_consensus.cc:697] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:54.242666 9659 consensus_queue.cc:237] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.244470 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:54.249197 9099 tablet_bootstrap.cc:654] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:54.250058 9523 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:48:54.256878 8740 leader_election.cc:343] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:54.257351 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:54.257472 9145 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:54.258082 9137 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:54.267599 9698 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 19:48:54.276744 9106 tablet_bootstrap.cc:654] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:54.285440 9137 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:54.288448 9101 tablet_bootstrap.cc:492] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:54.289049 9101 ts_tablet_manager.cc:1403] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.286s user 0.012s sys 0.007s
I20260807 19:48:54.310091 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:54.387185 9101 raft_consensus.cc:359] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.388937 9101 raft_consensus.cc:385] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:54.389197 9101 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:54.398864 9101 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.407428 8901 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:54.452656 9137 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:54.453310 9137 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.455425 9137 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:54.448186 8849 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:54.458385 9276 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:54.465924 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:48:54.479964 8740 leader_election.cc:343] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:54.495496 9101 ts_tablet_manager.cc:1434] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.206s user 0.008s sys 0.010s
W20260807 19:48:54.624476 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 124010us
I20260807 19:48:54.637172 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:48:54.768553 9723 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.149s user 0.000s sys 0.001s
I20260807 19:48:54.796146 9099 tablet_bootstrap.cc:492] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:54.796800 9099 ts_tablet_manager.cc:1403] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.603s user 0.024s sys 0.013s
I20260807 19:48:54.744200 9018 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:54.823599 9106 tablet_bootstrap.cc:492] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:54.824180 9106 ts_tablet_manager.cc:1403] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.605s user 0.018s sys 0.019s
I20260807 19:48:54.895371 9099 raft_consensus.cc:359] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.895968 9099 raft_consensus.cc:385] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:54.896193 9099 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:54.889292 9665 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:48:54.899446 9729 log.cc:927] Time spent T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.125s user 0.001s sys 0.000s
I20260807 19:48:54.904577 9665 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:54.905092 9665 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.910837 9101 tablet_bootstrap.cc:492] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:54.912545 9099 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.917186 9099 ts_tablet_manager.cc:1434] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.120s user 0.006s sys 0.000s
I20260807 19:48:54.918226 9099 tablet_bootstrap.cc:492] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:54.920252 9106 raft_consensus.cc:359] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.921211 9106 raft_consensus.cc:385] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:54.921479 9106 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:54.925271 9665 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:54.926769 9106 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:54.928514 9106 ts_tablet_manager.cc:1434] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.104s user 0.006s sys 0.000s
I20260807 19:48:54.929519 9106 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:54.934661 9099 tablet_bootstrap.cc:654] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:54.936920 9106 tablet_bootstrap.cc:654] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:54.936920 9101 tablet_bootstrap.cc:654] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.004496 9101 tablet_bootstrap.cc:492] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:55.005092 9101 ts_tablet_manager.cc:1403] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.101s user 0.010s sys 0.008s
W20260807 19:48:55.019721 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 58069.5us
I20260807 19:48:55.029410 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.030086 8790 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
W20260807 19:48:55.048153 9728 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.126s user 0.003s sys 0.000s
I20260807 19:48:55.063310 9752 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
W20260807 19:48:55.082438 8714 auto_rebalancer.cc:300] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260807 19:48:55.107959 9099 tablet_bootstrap.cc:492] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:55.109184 9099 ts_tablet_manager.cc:1403] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.191s user 0.032s sys 0.040s
I20260807 19:48:55.112421 9099 raft_consensus.cc:359] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.113214 9099 raft_consensus.cc:385] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.113457 9099 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.113924 9106 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:55.114509 9106 ts_tablet_manager.cc:1403] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.185s user 0.049s sys 0.042s
I20260807 19:48:55.113963 9099 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.116628 9099 ts_tablet_manager.cc:1434] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:48:55.118533 9099 tablet_bootstrap.cc:492] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:55.118201 9106 raft_consensus.cc:359] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.118968 9106 raft_consensus.cc:385] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.119230 9106 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.120402 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:48:55.119823 9106 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.121376 9662 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:48:55.121781 9662 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.122133 9662 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:55.122484 9106 ts_tablet_manager.cc:1434] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:48:55.123584 9106 tablet_bootstrap.cc:492] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:55.125082 9099 tablet_bootstrap.cc:654] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.129333 9106 tablet_bootstrap.cc:654] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.131521 9662 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:48:55.138688 8968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s user 0.040s sys 0.103s
W20260807 19:48:55.139117 8968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s user 0.040s sys 0.103s
W20260807 19:48:55.139382 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 862794us
W20260807 19:48:55.142207 8968 consensus_peers.cc:597] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Couldn't send request to peer 8e9ad2b5e2a94d4faff90af735c6d792. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:55.143406 9680 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:55.145253 8968 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:55.146780 9698 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260807 19:48:55.152554 8740 leader_election.cc:343] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:55.153023 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:55.162674 8628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38426 (request call id 130) took 1120 ms. Trace:
I20260807 19:48:55.166662 9106 tablet_bootstrap.cc:492] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:55.167258 9106 ts_tablet_manager.cc:1403] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.044s user 0.011s sys 0.003s
I20260807 19:48:55.170128 8632 catalog_manager.cc:5697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:55.172144 9709 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:55.174671 9753 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:55.176646 9688 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.177132 9688 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.179143 9688 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:55.181946 9760 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:55.182489 9106 raft_consensus.cc:359] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.183166 9106 raft_consensus.cc:385] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.183414 9106 raft_consensus.cc:740] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.184075 9106 consensus_queue.cc:260] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.185770 9106 ts_tablet_manager.cc:1434] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260807 19:48:55.187368 9106 tablet_bootstrap.cc:492] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:55.189328 9019 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.190094 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:55.204035 9106 tablet_bootstrap.cc:654] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.163371 8628 rpcz_store.cc:276] 0807 19:48:54.042098 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:48:54.042260 (+ 162us) service_pool.cc:224] Handling call
0807 19:48:55.162573 (+1120313us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:48:54.149014 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:48:54.149560 (+ 546us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:48:54.149597 (+ 37us) write_op.cc:435] Acquired schema lock
0807 19:48:54.149613 (+ 16us) tablet.cc:662] Decoding operations
0807 19:48:54.149973 (+ 360us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:48:54.150021 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0807 19:48:54.150048 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:48:54.150237 (+ 189us) tablet.cc:701] Row locks acquired
0807 19:48:54.150252 (+ 15us) write_op.cc:260] PREPARE: finished
0807 19:48:54.152308 (+ 2056us) write_op.cc:270] Start()
0807 19:48:54.152488 (+ 180us) write_op.cc:276] Timestamp: P: 1786132134150342 usec, L: 0
0807 19:48:54.155377 (+ 2889us) op_driver.cc:348] REPLICATION: starting
0807 19:48:54.183588 (+ 28211us) log.cc:844] Serialized 537 byte log entry
0807 19:48:54.258998 (+ 75410us) op_driver.cc:464] REPLICATION: finished
0807 19:48:54.615988 (+356990us) write_op.cc:301] APPLY: starting
0807 19:48:54.620440 (+ 4452us) tablet.cc:1367] starting BulkCheckPresence
0807 19:48:54.633033 (+ 12593us) tablet.cc:1370] finished BulkCheckPresence
0807 19:48:54.762761 (+129728us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:48:55.081524 (+318763us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b5c00196e0c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f68818304f1 00007f68818303e1 00007f688182b12c 00007f687ef70ceb 00007f687ef709b3 00007f687f06f814 00007f687f05b35c 00007f687f05f741 00007f687f05f6f9 00007f687f05f689 00007f687f05f651
0807 19:48:55.081813 (+ 289us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:48:55.082006 (+ 193us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:48:55.082056 (+ 50us) write_op.cc:312] APPLY: finished
0807 19:48:55.082309 (+ 253us) log.cc:844] Serialized 22 byte log entry
0807 19:48:55.091176 (+ 8867us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:48:55.091284 (+ 108us) write_op.cc:454] Released schema lock
0807 19:48:55.091324 (+ 40us) write_op.cc:341] FINISH: Updating metrics
0807 19:48:55.096372 (+ 5048us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:48:55.096408 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":230643,"apply.run_cpu_time_us":8402,"apply.run_wall_time_us":554717,"num_ops":1,"prepare.queue_time_us":14361,"prepare.run_cpu_time_us":3057,"prepare.run_wall_time_us":34868,"replication_time_us":100347,"spinlock_wait_cycles":23040896,"thread_start_us":51746,"threads_started":1}]]}
I20260807 19:48:55.219816 9662 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.229491 9760 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:55.201598 9101 raft_consensus.cc:359] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.232784 9101 raft_consensus.cc:385] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.233197 9101 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.234015 9101 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.236044 9099 tablet_bootstrap.cc:492] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:55.236620 9099 ts_tablet_manager.cc:1403] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.118s user 0.012s sys 0.000s
I20260807 19:48:55.236980 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:55.237625 8783 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:55.239513 9099 raft_consensus.cc:359] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.240240 9099 raft_consensus.cc:385] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.240622 9099 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.241274 9099 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.242959 9099 ts_tablet_manager.cc:1434] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 19:48:55.244002 9099 tablet_bootstrap.cc:492] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Bootstrap starting.
I20260807 19:48:55.257411 9101 ts_tablet_manager.cc:1434] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.252s user 0.007s sys 0.008s
I20260807 19:48:55.258795 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.275247 9145 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:48:55.275861 9145 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.278026 9276 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 19:48:55.285753 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.293859 9145 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:55.295048 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.297659 9101 tablet_bootstrap.cc:492] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:55.302237 8902 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 19:48:55.302630 8632 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38436 (request call id 114) took 1163 ms. Trace:
I20260807 19:48:55.303026 8632 rpcz_store.cc:276] 0807 19:48:54.138997 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:48:54.139152 (+ 155us) service_pool.cc:224] Handling call
0807 19:48:55.302565 (+1163413us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:48:55.243558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:48:55.244136 (+ 578us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:48:55.244165 (+ 29us) write_op.cc:435] Acquired schema lock
0807 19:48:55.245645 (+ 1480us) tablet.cc:662] Decoding operations
0807 19:48:55.247229 (+ 1584us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:48:55.247312 (+ 83us) write_op.cc:641] Partition lock acquired for write op
0807 19:48:55.247346 (+ 34us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:48:55.258339 (+ 10993us) tablet.cc:701] Row locks acquired
0807 19:48:55.258557 (+ 218us) write_op.cc:260] PREPARE: finished
0807 19:48:55.258703 (+ 146us) write_op.cc:270] Start()
0807 19:48:55.258818 (+ 115us) write_op.cc:276] Timestamp: P: 1786132135258664 usec, L: 0
0807 19:48:55.258849 (+ 31us) op_driver.cc:348] REPLICATION: starting
0807 19:48:55.262001 (+ 3152us) log.cc:844] Serialized 537 byte log entry
0807 19:48:55.298885 (+ 36884us) op_driver.cc:464] REPLICATION: finished
0807 19:48:55.299221 (+ 336us) write_op.cc:301] APPLY: starting
0807 19:48:55.299299 (+ 78us) tablet.cc:1367] starting BulkCheckPresence
0807 19:48:55.299409 (+ 110us) tablet.cc:1370] finished BulkCheckPresence
0807 19:48:55.299506 (+ 97us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:48:55.299755 (+ 249us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:48:55.299942 (+ 187us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:48:55.299996 (+ 54us) write_op.cc:312] APPLY: finished
0807 19:48:55.300210 (+ 214us) log.cc:844] Serialized 22 byte log entry
0807 19:48:55.300392 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:48:55.300477 (+ 85us) write_op.cc:454] Released schema lock
0807 19:48:55.300515 (+ 38us) write_op.cc:341] FINISH: Updating metrics
0807 19:48:55.300935 (+ 420us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:48:55.300960 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":121,"apply.run_cpu_time_us":1611,"apply.run_wall_time_us":1686,"num_ops":1,"prepare.queue_time_us":69783,"prepare.run_cpu_time_us":3082,"prepare.run_wall_time_us":20299,"replication_time_us":39879,"thread_start_us":70838,"threads_started":1}]]}
I20260807 19:48:55.307046 9099 tablet_bootstrap.cc:654] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.307075 8629 catalog_manager.cc:5697] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:55.327059 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:55.337134 8783 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:48:55.338414 8778 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260807 19:48:55.347261 9688 consensus_queue.cc:1059] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:55.347944 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.348645 9760 raft_consensus.cc:3055] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:48:55.349063 9760 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:55.349902 9760 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.351074 9760 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:48:55.353295 8972 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:55.355607 9145 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:48:55.357192 9106 tablet_bootstrap.cc:492] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:55.357636 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:55.357681 9106 ts_tablet_manager.cc:1403] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.171s user 0.030s sys 0.024s
I20260807 19:48:55.359961 9276 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:48:55.360965 9276 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:55.361764 9276 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.372254 8972 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:55.377274 9101 tablet_bootstrap.cc:654] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.382640 9145 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:48:55.439205 9099 tablet_bootstrap.cc:492] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: No bootstrap required, opened a new log
I20260807 19:48:55.439752 9099 ts_tablet_manager.cc:1403] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent bootstrapping tablet: real 0.196s user 0.007s sys 0.007s
W20260807 19:48:55.441898 9780 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260807 19:48:55.442270 9106 raft_consensus.cc:359] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.443019 9106 raft_consensus.cc:385] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.443287 9106 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.443354 9099 raft_consensus.cc:359] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.444022 9099 raft_consensus.cc:385] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.444303 9099 raft_consensus.cc:740] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.451647 9099 consensus_queue.cc:260] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.463769 9099 ts_tablet_manager.cc:1434] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent starting tablet: real 0.024s user 0.010s sys 0.000s
I20260807 19:48:55.474644 9753 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.475164 9753 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.477965 9753 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.478581 9744 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.479161 9744 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.481329 9744 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.487401 9106 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.495361 9106 ts_tablet_manager.cc:1434] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.137s user 0.007s sys 0.002s
I20260807 19:48:55.498202 9106 tablet_bootstrap.cc:492] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:55.515169 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:55.515934 9023 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:55.519217 8778 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.521565 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:55.524386 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.525077 8903 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:55.527462 9101 tablet_bootstrap.cc:492] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:55.528028 9101 ts_tablet_manager.cc:1403] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.235s user 0.014s sys 0.008s
I20260807 19:48:55.530686 9101 raft_consensus.cc:359] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.531425 9101 raft_consensus.cc:385] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.531744 9101 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.532646 9101 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.534267 9106 tablet_bootstrap.cc:654] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.534420 9101 ts_tablet_manager.cc:1434] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260807 19:48:55.534451 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.535722 9101 tablet_bootstrap.cc:492] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:55.542277 9101 tablet_bootstrap.cc:654] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.548413 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.552464 9744 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:55.552902 9744 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.553241 9744 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.557264 9101 tablet_bootstrap.cc:492] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:55.558477 9101 ts_tablet_manager.cc:1403] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.006s
I20260807 19:48:55.558133 8632 catalog_manager.cc:5697] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:55.560505 9744 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.562497 9744 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.566259 9101 raft_consensus.cc:359] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.567083 9101 raft_consensus.cc:385] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.567827 9101 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.568683 9101 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.571612 9101 ts_tablet_manager.cc:1434] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260807 19:48:55.572791 9101 tablet_bootstrap.cc:492] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4: Bootstrap starting.
I20260807 19:48:55.574585 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:55.575192 9017 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.577160 9106 tablet_bootstrap.cc:492] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:55.577672 9106 ts_tablet_manager.cc:1403] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.080s user 0.009s sys 0.002s
I20260807 19:48:55.579658 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:55.584583 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:55.585367 8906 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.586149 9017 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:55.586839 9276 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.587292 9276 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.589334 9276 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.590778 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:55.591886 9744 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:48:55.593760 8906 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:48:55.594424 9753 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.601408 9753 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.595621 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.602497 8779 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:55.598914 9106 raft_consensus.cc:359] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.603284 9106 raft_consensus.cc:385] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.603513 9753 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.603649 9106 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.604516 9106 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.606244 9023 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:55.607730 9101 tablet_bootstrap.cc:654] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.608594 8856 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.608311 9106 ts_tablet_manager.cc:1434] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.030s user 0.008s sys 0.000s
I20260807 19:48:55.609360 9276 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:55.609724 9276 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.610037 9276 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.610574 9106 tablet_bootstrap.cc:492] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Bootstrap starting.
I20260807 19:48:55.621840 9106 tablet_bootstrap.cc:654] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Neither blocks nor log segments found. Creating new log.
I20260807 19:48:55.596366 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.630146 9101 tablet_bootstrap.cc:492] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4: No bootstrap required, opened a new log
I20260807 19:48:55.632925 9101 ts_tablet_manager.cc:1403] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4: Time spent bootstrapping tablet: real 0.060s user 0.014s sys 0.001s
I20260807 19:48:55.635845 9753 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:55.636195 9145 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.636521 9753 raft_consensus.cc:697] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:48:55.636813 9145 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.637403 9753 consensus_queue.cc:237] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.640049 9145 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:55.641036 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.641714 8779 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:55.642091 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.643159 8968 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.643997 9145 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 19:48:55.644124 8972 leader_election.cc:343] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Illegal state: must be running to vote when last-logged opid is not known
I20260807 19:48:55.644502 9145 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.644862 9145 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.645172 9101 raft_consensus.cc:359] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.645951 9101 raft_consensus.cc:385] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.646278 9101 raft_consensus.cc:740] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.647127 9101 consensus_queue.cc:260] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.649873 9101 ts_tablet_manager.cc:1434] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
W20260807 19:48:55.651731 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 55468.9us
I20260807 19:48:55.653496 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.654211 9145 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.656292 9145 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:55.658313 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:55.658969 8778 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.659104 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:55.659773 9023 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 0.
I20260807 19:48:55.661252 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:55.661940 8913 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.666477 8778 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:55.668025 8968 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.669010 9145 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:55.670138 9106 tablet_bootstrap.cc:492] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: No bootstrap required, opened a new log
I20260807 19:48:55.670753 9106 ts_tablet_manager.cc:1403] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Time spent bootstrapping tablet: real 0.061s user 0.013s sys 0.005s
I20260807 19:48:55.674140 9276 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.675184 8910 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:55.670183 9145 raft_consensus.cc:697] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:55.678045 9145 consensus_queue.cc:237] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.677431 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:55.683425 9023 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.689589 9106 raft_consensus.cc:359] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.690444 9106 raft_consensus.cc:385] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:48:55.690793 9106 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Initialized, Role: FOLLOWER
I20260807 19:48:55.692026 9106 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.694700 9106 ts_tablet_manager.cc:1434] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Time spent starting tablet: real 0.024s user 0.006s sys 0.000s
I20260807 19:48:55.701924 8632 catalog_manager.cc:5697] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:55.702590 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.703645 9753 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:55.704020 9753 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.704387 9753 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.716310 9753 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.719319 9753 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.717451 8629 catalog_manager.cc:5697] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:55.724335 9276 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.724525 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:55.723639 8913 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:55.727591 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:55.728466 8778 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 1: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:48:55.691157 9023 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:55.740525 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:55.741473 8906 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 1: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:48:55.747119 8856 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:48:55.747119 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:48:55.748977 9675 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:55.749541 9675 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:55.750384 9675 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.750324 9753 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 19:48:55.772326 8630 catalog_manager.cc:5697] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:55.840956 8778 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260807 19:48:55.840386 9023 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260807 19:48:55.841672 9023 pending_rounds.cc:85] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Aborting all ops after (but not including) 0
I20260807 19:48:55.841965 9023 pending_rounds.cc:107] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260807 19:48:55.842260 9023 raft_consensus.cc:2889] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:48:55.842264 9276 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 19:48:55.844237 9665 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:55.891839 9688 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.892431 9688 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.894227 9688 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:55.895062 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.895493 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:48:55.895754 8778 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:55.896080 8910 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 0.
I20260807 19:48:55.896806 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.897460 9688 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:55.897755 9688 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.898031 9688 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.903291 9688 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.905084 9688 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:55.905927 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:55.906091 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:48:55.906574 8778 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.906641 8910 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.912438 8778 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:55.912943 8910 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:48:55.913750 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.914580 9760 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:55.915048 9760 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:48:55.915868 9760 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.930299 8632 catalog_manager.cc:5697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 0 to 1, leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:48:55.945587 9276 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:48:55.946110 9276 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.947642 9276 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.948652 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:48:55.948587 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:48:55.949399 9021 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:55.949491 8779 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 0.
I20260807 19:48:55.950428 8856 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.951203 9276 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:48:55.951540 9276 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:48:55.951893 9276 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.956710 9276 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.958323 9276 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:48:55.959023 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:48:55.959174 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:48:55.959519 8778 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.959653 9021 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:48:55.964022 9021 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:55.964164 8778 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:48:55.965128 8856 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:48:55.965768 9276 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:48:55.966223 9276 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:48:55.966796 9276 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:48:55.981200 8632 catalog_manager.cc:5697] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:56.028698 9482 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.041478 9753 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.109730 9760 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.127696 9145 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.233349 9665 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.253902 9276 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.304661 9145 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.314723 9760 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.394793 9276 consensus_queue.cc:1048] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:56.405314 9276 consensus_queue.cc:1048] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:48:57.694782 6315 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260807 19:48:57.704900 9827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:57.704937 9826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:48:57.707633 9830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:48:57.708501 6315 server_base.cc:1061] running on GCE node
I20260807 19:48:57.710876 6315 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260807 19:48:57.711231 6315 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260807 19:48:57.711511 6315 hybrid_clock.cc:648] HybridClock initialized: now 1786132137711489 us; error 0 us; skew 500 ppm
I20260807 19:48:57.712451 6315 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:48:57.715653 6315 webserver.cc:533] Webserver started at http://127.6.42.196:38691/ using document root <none> and password file <none>
I20260807 19:48:57.716286 6315 fs_manager.cc:362] Metadata directory not provided
I20260807 19:48:57.716576 6315 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:48:57.716922 6315 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:48:57.718767 6315 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-3-root/instance:
uuid: "39975243456d4452b4a4870ac7ebafc5"
format_stamp: "Formatted at 2026-08-07 19:48:57 on dist-test-slave-lkx2"
I20260807 19:48:57.725169 6315 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:48:57.729894 9837 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:57.730875 6315 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:48:57.731245 6315 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-3-root
uuid: "39975243456d4452b4a4870ac7ebafc5"
format_stamp: "Formatted at 2026-08-07 19:48:57 on dist-test-slave-lkx2"
I20260807 19:48:57.731559 6315 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:48:57.751529 6315 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260807 19:48:57.753466 6315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:48:57.758765 6315 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260807 19:48:57.761718 6315 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:48:57.762023 6315 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:57.762358 6315 ts_tablet_manager.cc:616] Registered 0 tablets
I20260807 19:48:57.762569 6315 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:48:57.848924 6315 rpc_server.cc:307] RPC server started. Bound to: 127.6.42.196:35143
I20260807 19:48:57.849043 9937 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.42.196:35143 every 8 connection(s)
I20260807 19:48:57.853425 6315 auto_leader_rebalancer-test.cc:581] add a tserver: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:48:57.897680 9938 heartbeater.cc:344] Connected to a master server at 127.6.42.254:41447
I20260807 19:48:57.898204 9938 heartbeater.cc:461] Registering TS with master...
I20260807 19:48:57.899170 9938 heartbeater.cc:507] Master 127.6.42.254:41447 requested a full tablet report, sending...
I20260807 19:48:57.901145 8629 ts_manager.cc:194] Registered new tserver with Master: 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:48:57.902676 8629 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59414
I20260807 19:48:58.320520 8913 consensus_queue.cc:237] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:58.330439 9021 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.331454 8779 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.332332 9943 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:48:58.335587 9276 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:48:58.355356 8853 consensus_peers.cc:597] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92d07404635f47998c0f61e0e7f5b732. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:58.360278 9675 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.364142 8779 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.364804 9017 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.380903 8629 catalog_manager.cc:5697] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.405635 8779 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:58.417286 9017 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.417490 8913 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.419390 9935 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:58.421434 9935 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:48:58.431244 8737 consensus_peers.cc:597] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2fe2de08a4ba49eea927f371b4bf0e5f. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:58.430565 9482 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.432940 9017 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.435011 8902 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.445188 8913 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:58.453115 8630 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
W20260807 19:48:58.473598 8853 consensus_peers.cc:597] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 138f0e8a31994c549dd743c9ba49d6ee. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:58.474841 9021 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.476933 9955 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:58.483845 8779 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.487866 9955 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:58.493180 9675 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.495644 9021 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.507726 8779 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.510475 8778 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:58.514006 8629 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:58.524691 8913 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 19:48:58.525106 8737 consensus_peers.cc:597] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf699a9379e64939a7048af91aef93ec. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:48:58.524840 9021 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:48:58.526538 9977 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:58.527673 9482 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:48:58.548506 9935 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.551689 8902 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.557818 9021 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:48:58.572608 8632 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:48:58.843626 9988 ts_tablet_manager.cc:933] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:58.846176 9988 tablet_copy_client.cc:323] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.194:37659
I20260807 19:48:58.860679 8928 tablet_copy_service.cc:140] P 671d8754170646eaac5748745eba58b4: Received BeginTabletCopySession request for tablet 92d07404635f47998c0f61e0e7f5b732 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53310)
I20260807 19:48:58.861301 8928 tablet_copy_service.cc:161] P 671d8754170646eaac5748745eba58b4: Beginning new tablet copy session on tablet 92d07404635f47998c0f61e0e7f5b732 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53310: session id = 39975243456d4452b4a4870ac7ebafc5-92d07404635f47998c0f61e0e7f5b732
I20260807 19:48:58.869259 8928 tablet_copy_source_session.cc:215] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:48:58.873399 9988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d07404635f47998c0f61e0e7f5b732. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:58.889456 9988 tablet_copy_client.cc:806] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:48:58.890211 9988 tablet_copy_client.cc:670] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:48:58.901432 9988 tablet_copy_client.cc:538] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:48:58.905555 9938 heartbeater.cc:499] Master 127.6.42.254:41447 was elected leader, sending a full tablet report...
I20260807 19:48:58.912781 9988 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:48:58.940833 9988 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:48:58.941840 9988 tablet_bootstrap.cc:492] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:48:58.942682 9988 ts_tablet_manager.cc:1403] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.030s user 0.013s sys 0.011s
I20260807 19:48:58.946293 9988 raft_consensus.cc:359] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:58.947319 9988 raft_consensus.cc:740] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:48:58.948113 9988 consensus_queue.cc:260] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:58.959848 9996 ts_tablet_manager.cc:933] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
I20260807 19:48:58.960484 9988 ts_tablet_manager.cc:1434] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260807 19:48:58.968293 8928 tablet_copy_service.cc:342] P 671d8754170646eaac5748745eba58b4: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-92d07404635f47998c0f61e0e7f5b732 received from {username='slave'} at 127.0.0.1:53310
I20260807 19:48:58.968892 8928 tablet_copy_service.cc:434] P 671d8754170646eaac5748745eba58b4: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-92d07404635f47998c0f61e0e7f5b732 on tablet 92d07404635f47998c0f61e0e7f5b732 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:48:58.968930 9996 tablet_copy_client.cc:323] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.193:33185
I20260807 19:48:58.982388 8805 tablet_copy_service.cc:140] P 8e9ad2b5e2a94d4faff90af735c6d792: Received BeginTabletCopySession request for tablet 2fe2de08a4ba49eea927f371b4bf0e5f from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53512)
I20260807 19:48:58.982988 8805 tablet_copy_service.cc:161] P 8e9ad2b5e2a94d4faff90af735c6d792: Beginning new tablet copy session on tablet 2fe2de08a4ba49eea927f371b4bf0e5f from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53512: session id = 39975243456d4452b4a4870ac7ebafc5-2fe2de08a4ba49eea927f371b4bf0e5f
I20260807 19:48:58.992564 8805 tablet_copy_source_session.cc:215] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:48:58.996192 9996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fe2de08a4ba49eea927f371b4bf0e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:59.020601 9996 tablet_copy_client.cc:806] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:48:59.021270 9996 tablet_copy_client.cc:670] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:48:59.021957 9988 ts_tablet_manager.cc:933] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:48:59.030503 9988 tablet_copy_client.cc:323] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.194:37659
I20260807 19:48:59.032189 8928 tablet_copy_service.cc:140] P 671d8754170646eaac5748745eba58b4: Received BeginTabletCopySession request for tablet 138f0e8a31994c549dd743c9ba49d6ee from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53310)
I20260807 19:48:59.032733 8928 tablet_copy_service.cc:161] P 671d8754170646eaac5748745eba58b4: Beginning new tablet copy session on tablet 138f0e8a31994c549dd743c9ba49d6ee from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53310: session id = 39975243456d4452b4a4870ac7ebafc5-138f0e8a31994c549dd743c9ba49d6ee
I20260807 19:48:59.038354 8928 tablet_copy_source_session.cc:215] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:48:59.042618 9988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 138f0e8a31994c549dd743c9ba49d6ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:59.049075 9996 tablet_copy_client.cc:538] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:48:59.058434 9996 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:48:59.061940 9988 tablet_copy_client.cc:806] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:48:59.063235 9988 tablet_copy_client.cc:670] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:48:59.066628 10005 ts_tablet_manager.cc:933] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
I20260807 19:48:59.076277 9988 tablet_copy_client.cc:538] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:48:59.087253 9988 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:48:59.104296 9996 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:48:59.106323 9996 tablet_bootstrap.cc:492] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:48:59.107081 9996 ts_tablet_manager.cc:1403] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.049s user 0.022s sys 0.012s
I20260807 19:48:59.113394 10005 tablet_copy_client.cc:323] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.193:33185
I20260807 19:48:59.115686 9996 raft_consensus.cc:359] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:59.116637 9996 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:48:59.117336 9996 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:59.118872 8805 tablet_copy_service.cc:140] P 8e9ad2b5e2a94d4faff90af735c6d792: Received BeginTabletCopySession request for tablet bf699a9379e64939a7048af91aef93ec from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53512)
I20260807 19:48:59.119433 8805 tablet_copy_service.cc:161] P 8e9ad2b5e2a94d4faff90af735c6d792: Beginning new tablet copy session on tablet bf699a9379e64939a7048af91aef93ec from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53512: session id = 39975243456d4452b4a4870ac7ebafc5-bf699a9379e64939a7048af91aef93ec
I20260807 19:48:59.119742 9996 ts_tablet_manager.cc:1434] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.012s user 0.006s sys 0.002s
I20260807 19:48:59.122509 8802 tablet_copy_service.cc:342] P 8e9ad2b5e2a94d4faff90af735c6d792: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-2fe2de08a4ba49eea927f371b4bf0e5f received from {username='slave'} at 127.0.0.1:53512
I20260807 19:48:59.122975 8802 tablet_copy_service.cc:434] P 8e9ad2b5e2a94d4faff90af735c6d792: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-2fe2de08a4ba49eea927f371b4bf0e5f on tablet 2fe2de08a4ba49eea927f371b4bf0e5f with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:48:59.127872 8805 tablet_copy_source_session.cc:215] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:48:59.129072 9988 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:48:59.130318 9988 tablet_bootstrap.cc:492] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:48:59.131023 9988 ts_tablet_manager.cc:1403] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.044s user 0.014s sys 0.007s
I20260807 19:48:59.134306 9988 raft_consensus.cc:359] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:59.135181 9988 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:48:59.135617 10005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf699a9379e64939a7048af91aef93ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:48:59.135846 9988 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:59.138268 9988 ts_tablet_manager.cc:1434] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 19:48:59.140548 8928 tablet_copy_service.cc:342] P 671d8754170646eaac5748745eba58b4: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-138f0e8a31994c549dd743c9ba49d6ee received from {username='slave'} at 127.0.0.1:53310
I20260807 19:48:59.141002 8928 tablet_copy_service.cc:434] P 671d8754170646eaac5748745eba58b4: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-138f0e8a31994c549dd743c9ba49d6ee on tablet 138f0e8a31994c549dd743c9ba49d6ee with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:48:59.164209 10005 tablet_copy_client.cc:806] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:48:59.165009 10005 tablet_copy_client.cc:670] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:48:59.170097 10005 tablet_copy_client.cc:538] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:48:59.180233 10005 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:48:59.203965 10005 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:48:59.204910 10005 tablet_bootstrap.cc:492] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:48:59.205605 10005 ts_tablet_manager.cc:1403] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.026s user 0.011s sys 0.015s
I20260807 19:48:59.209316 10005 raft_consensus.cc:359] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:59.210187 10005 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:48:59.210829 10005 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:48:59.213171 10005 ts_tablet_manager.cc:1434] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260807 19:48:59.214972 8805 tablet_copy_service.cc:342] P 8e9ad2b5e2a94d4faff90af735c6d792: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-bf699a9379e64939a7048af91aef93ec received from {username='slave'} at 127.0.0.1:53512
I20260807 19:48:59.215442 8805 tablet_copy_service.cc:434] P 8e9ad2b5e2a94d4faff90af735c6d792: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-bf699a9379e64939a7048af91aef93ec on tablet bf699a9379e64939a7048af91aef93ec with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:48:59.243052 9898 raft_consensus.cc:1217] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:48:59.437299 9898 raft_consensus.cc:1217] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:48:59.538072 9898 raft_consensus.cc:1217] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:48:59.551335 9898 raft_consensus.cc:1217] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:48:59.621649 9675 raft_consensus.cc:1064] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:48:59.624583 9675 consensus_queue.cc:237] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:48:59.632054 9017 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:48:59.632905 8778 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:48:59.632947 9898 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:48:59.634333 9943 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:59.635284 9675 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:59.636025 10003 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:59.673532 10003 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.681744 9898 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.689558 9017 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.681746 8778 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.697228 8629 catalog_manager.cc:5697] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.752146 8902 consensus_queue.cc:237] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:48:59.758617 9898 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:48:59.760100 9023 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:48:59.762729 10021 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:59.764295 10020 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:48:59.775867 10003 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.787388 9021 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.791003 9898 raft_consensus.cc:2955] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.796674 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 92d07404635f47998c0f61e0e7f5b732 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:48:59.805881 8632 catalog_manager.cc:5697] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 reported cstate change: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:48:59.827876 8769 tablet_service.cc:1558] Processing DeleteTablet for tablet 92d07404635f47998c0f61e0e7f5b732 with delete_type TABLET_DATA_TOMBSTONED (TS 8e9ad2b5e2a94d4faff90af735c6d792 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60644
I20260807 19:48:59.854267 10029 tablet_replica.cc:333] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: stopping tablet replica
I20260807 19:48:59.855466 10029 raft_consensus.cc:2243] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:48:59.856796 10029 raft_consensus.cc:2272] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:48:59.860714 10029 ts_tablet_manager.cc:1916] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:48:59.876953 10029 ts_tablet_manager.cc:1929] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:48:59.877449 10029 log.cc:1199] T 92d07404635f47998c0f61e0e7f5b732 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/wals/92d07404635f47998c0f61e0e7f5b732
I20260807 19:48:59.879473 8613 catalog_manager.cc:5028] TS 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): tablet 92d07404635f47998c0f61e0e7f5b732 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:48:59.921571 9977 raft_consensus.cc:1064] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:48:59.924219 9977 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:48:59.934315 9898 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:48:59.937671 9482 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:59.943542 9021 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:48:59.944940 8910 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:48:59.948908 9482 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:59.955168 9482 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:48:59.963724 9977 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.965583 9021 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.966506 8910 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.993139 9898 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:48:59.995226 8632 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.029552 8778 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:00.038758 9897 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:00.040271 9017 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:00.041272 9935 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:00.042203 9977 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:00.045435 9977 raft_consensus.cc:1064] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:00.048214 9977 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:00.052311 9482 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.062070 8613 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2fe2de08a4ba49eea927f371b4bf0e5f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:00.064941 9898 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.072152 8628 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:00.073704 9897 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:00.078192 10042 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:00.070381 9017 raft_consensus.cc:2955] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.081192 8910 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:00.075996 9021 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:00.086036 9977 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:00.096081 8893 tablet_service.cc:1558] Processing DeleteTablet for tablet 2fe2de08a4ba49eea927f371b4bf0e5f with delete_type TABLET_DATA_TOMBSTONED (TS 671d8754170646eaac5748745eba58b4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39358
I20260807 19:49:00.103663 9935 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:00.124627 10046 tablet_replica.cc:333] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: stopping tablet replica
I20260807 19:49:00.155937 10003 raft_consensus.cc:1064] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:00.158915 10003 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:00.169976 10046 raft_consensus.cc:2243] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:00.170943 10046 raft_consensus.cc:2272] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:00.175256 9482 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.177876 9017 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.184001 10046 ts_tablet_manager.cc:1916] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:00.241995 9898 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.251173 8628 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.259183 8910 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.287071 8778 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:00.298303 9897 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:00.305747 9935 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:00.309351 9897 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:00.310694 9017 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:00.314023 10003 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:00.315164 10020 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:00.319507 8780 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:00.322619 9898 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.319204 10042 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.324532 8906 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:00.333009 9675 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:00.335283 9935 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:00.343101 8906 raft_consensus.cc:2955] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.376049 8613 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bf699a9379e64939a7048af91aef93ec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:00.387024 8630 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:00.387491 10020 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.393209 9017 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.394657 8779 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.410300 9898 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.412075 8996 tablet_service.cc:1558] Processing DeleteTablet for tablet bf699a9379e64939a7048af91aef93ec with delete_type TABLET_DATA_TOMBSTONED (TS a38f4d6088cf4c8cb510200ca0555801 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52256
I20260807 19:49:00.388705 10046 ts_tablet_manager.cc:1929] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:00.435024 10059 tablet_replica.cc:333] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: stopping tablet replica
I20260807 19:49:00.436553 10046 log.cc:1199] T 2fe2de08a4ba49eea927f371b4bf0e5f P 671d8754170646eaac5748745eba58b4: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root/wals/2fe2de08a4ba49eea927f371b4bf0e5f
I20260807 19:49:00.445077 8632 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.451406 8616 catalog_manager.cc:5028] TS 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): tablet 2fe2de08a4ba49eea927f371b4bf0e5f (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:00.454916 10059 raft_consensus.cc:2243] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:00.455852 10059 raft_consensus.cc:2272] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:00.460755 10059 ts_tablet_manager.cc:1916] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:00.482477 10059 ts_tablet_manager.cc:1929] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:00.482977 10059 log.cc:1199] T bf699a9379e64939a7048af91aef93ec P a38f4d6088cf4c8cb510200ca0555801: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root/wals/bf699a9379e64939a7048af91aef93ec
I20260807 19:49:00.485523 8616 catalog_manager.cc:5028] TS a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): tablet bf699a9379e64939a7048af91aef93ec (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:00.497399 8910 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:00.509783 9898 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:00.511690 10020 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:00.518626 8779 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:00.520658 10020 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:00.533012 9675 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.535130 9898 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.539158 8779 raft_consensus.cc:2955] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.550990 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 138f0e8a31994c549dd743c9ba49d6ee with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:00.552526 8632 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:00.567994 8996 tablet_service.cc:1558] Processing DeleteTablet for tablet 138f0e8a31994c549dd743c9ba49d6ee with delete_type TABLET_DATA_TOMBSTONED (TS a38f4d6088cf4c8cb510200ca0555801 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52256
I20260807 19:49:00.569262 10059 tablet_replica.cc:333] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: stopping tablet replica
I20260807 19:49:00.570196 10059 raft_consensus.cc:2243] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:00.570991 10059 raft_consensus.cc:2272] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:00.574522 10059 ts_tablet_manager.cc:1916] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:00.592514 10059 ts_tablet_manager.cc:1929] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:00.593343 10059 log.cc:1199] T 138f0e8a31994c549dd743c9ba49d6ee P a38f4d6088cf4c8cb510200ca0555801: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root/wals/138f0e8a31994c549dd743c9ba49d6ee
I20260807 19:49:00.595124 8616 catalog_manager.cc:5028] TS a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): tablet 138f0e8a31994c549dd743c9ba49d6ee (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:01.657759 9017 consensus_queue.cc:237] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:01.665555 8906 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.667420 8779 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.667358 10054 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:01.669245 10012 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:01.679237 10012 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
W20260807 19:49:01.682395 8970 consensus_peers.cc:597] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 413683c7378447239874762d812d75e5. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:01.682264 8906 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.682448 8779 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.696317 9017 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:01.699393 8628 catalog_manager.cc:5697] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.716050 8779 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.716050 8906 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.717746 10054 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:49:01.718663 8970 consensus_peers.cc:597] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be133eef91634be592094b95357ee200. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:01.719420 10012 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:01.731539 10012 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.733368 8780 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.746702 8778 consensus_queue.cc:237] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:01.754817 8910 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.758360 8632 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:01.769397 9021 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 19:49:01.774880 8737 consensus_peers.cc:597] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab8e617d098d425cbddc5d78da43ab99. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:01.776053 8906 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.778677 10060 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:01.795845 10060 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260807 19:49:01.816776 10036 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.833384 8910 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.843497 9021 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.845309 9017 consensus_queue.cc:237] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:01.856518 8630 catalog_manager.cc:5697] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260807 19:49:01.875842 8970 consensus_peers.cc:597] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb33dc9f5a49409bb8817073ec1143da. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:01.877846 8778 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.879377 8910 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:01.881232 10080 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:01.882180 10012 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:01.896166 10012 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.898105 8910 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.900333 8778 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:01.914885 8632 catalog_manager.cc:5697] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:02.185500 10107 ts_tablet_manager.cc:933] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:02.188248 10107 tablet_copy_client.cc:323] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
I20260807 19:49:02.211899 10115 ts_tablet_manager.cc:933] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
I20260807 19:49:02.216535 10115 tablet_copy_client.cc:323] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.193:33185
I20260807 19:49:02.217361 9044 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet 413683c7378447239874762d812d75e5 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:02.218075 9044 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet 413683c7378447239874762d812d75e5 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-413683c7378447239874762d812d75e5
I20260807 19:49:02.218520 8805 tablet_copy_service.cc:140] P 8e9ad2b5e2a94d4faff90af735c6d792: Received BeginTabletCopySession request for tablet ab8e617d098d425cbddc5d78da43ab99 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53512)
I20260807 19:49:02.219079 8805 tablet_copy_service.cc:161] P 8e9ad2b5e2a94d4faff90af735c6d792: Beginning new tablet copy session on tablet ab8e617d098d425cbddc5d78da43ab99 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53512: session id = 39975243456d4452b4a4870ac7ebafc5-ab8e617d098d425cbddc5d78da43ab99
I20260807 19:49:02.226504 9044 tablet_copy_source_session.cc:215] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:02.229568 10107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 413683c7378447239874762d812d75e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.229609 8805 tablet_copy_source_session.cc:215] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:02.233192 10115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab8e617d098d425cbddc5d78da43ab99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.250156 10107 tablet_copy_client.cc:806] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:02.250801 10107 tablet_copy_client.cc:670] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:02.251361 10115 tablet_copy_client.cc:806] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:02.251874 10115 tablet_copy_client.cc:670] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:02.254943 10107 tablet_copy_client.cc:538] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:02.255812 10115 tablet_copy_client.cc:538] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:02.264609 10107 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:02.268770 10115 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:02.297927 10120 ts_tablet_manager.cc:933] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:02.308738 10107 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:02.315765 10107 tablet_bootstrap.cc:492] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:02.318419 10107 ts_tablet_manager.cc:1403] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.054s user 0.028s sys 0.005s
I20260807 19:49:02.318504 10115 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:02.319563 10115 tablet_bootstrap.cc:492] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:02.320261 10115 ts_tablet_manager.cc:1403] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.052s user 0.033s sys 0.007s
I20260807 19:49:02.321844 10107 raft_consensus.cc:359] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.323299 10115 raft_consensus.cc:359] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.322717 10107 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:02.324167 10115 raft_consensus.cc:740] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:02.324939 10107 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.325461 10115 consensus_queue.cc:260] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.385617 10107 ts_tablet_manager.cc:1434] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.067s user 0.012s sys 0.012s
I20260807 19:49:02.388113 9044 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-413683c7378447239874762d812d75e5 received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:02.388885 9044 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-413683c7378447239874762d812d75e5 on tablet 413683c7378447239874762d812d75e5 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:02.417943 10115 ts_tablet_manager.cc:1434] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.097s user 0.026s sys 0.007s
I20260807 19:49:02.428594 8805 tablet_copy_service.cc:342] P 8e9ad2b5e2a94d4faff90af735c6d792: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-ab8e617d098d425cbddc5d78da43ab99 received from {username='slave'} at 127.0.0.1:53512
I20260807 19:49:02.429126 8805 tablet_copy_service.cc:434] P 8e9ad2b5e2a94d4faff90af735c6d792: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-ab8e617d098d425cbddc5d78da43ab99 on tablet ab8e617d098d425cbddc5d78da43ab99 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:02.434218 10107 ts_tablet_manager.cc:933] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:02.438416 10107 tablet_copy_client.cc:323] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
I20260807 19:49:02.423853 10120 tablet_copy_client.cc:323] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
I20260807 19:49:02.444638 9044 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet fb33dc9f5a49409bb8817073ec1143da from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:02.445075 9042 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet be133eef91634be592094b95357ee200 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:02.445294 9044 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet fb33dc9f5a49409bb8817073ec1143da from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-fb33dc9f5a49409bb8817073ec1143da
I20260807 19:49:02.446635 9042 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet be133eef91634be592094b95357ee200 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-be133eef91634be592094b95357ee200
I20260807 19:49:02.453938 9044 tablet_copy_source_session.cc:215] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:02.457437 10107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb33dc9f5a49409bb8817073ec1143da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.470026 9042 tablet_copy_source_session.cc:215] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:02.473234 10120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be133eef91634be592094b95357ee200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:02.482429 10107 tablet_copy_client.cc:806] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:02.483119 10107 tablet_copy_client.cc:670] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:02.487538 10107 tablet_copy_client.cc:538] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:02.503165 10107 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:02.509882 10120 tablet_copy_client.cc:806] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:02.510591 10120 tablet_copy_client.cc:670] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:02.516559 10120 tablet_copy_client.cc:538] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:02.610977 10107 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:02.613337 10107 tablet_bootstrap.cc:492] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:02.614591 10107 ts_tablet_manager.cc:1403] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.112s user 0.012s sys 0.014s
I20260807 19:49:02.648949 10120 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:02.748427 9898 raft_consensus.cc:1217] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:02.754722 10107 raft_consensus.cc:359] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.766016 10107 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:02.774466 10107 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.780884 9898 raft_consensus.cc:1217] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:02.785415 10107 ts_tablet_manager.cc:1434] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.170s user 0.008s sys 0.004s
I20260807 19:49:02.788555 9042 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-fb33dc9f5a49409bb8817073ec1143da received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:02.789270 9042 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-fb33dc9f5a49409bb8817073ec1143da on tablet fb33dc9f5a49409bb8817073ec1143da with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:02.864167 10120 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:02.865610 10120 tablet_bootstrap.cc:492] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:02.866487 10120 ts_tablet_manager.cc:1403] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.218s user 0.025s sys 0.008s
I20260807 19:49:02.869863 10120 raft_consensus.cc:359] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.870913 10120 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:02.872249 10120 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:02.876128 10120 ts_tablet_manager.cc:1434] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:49:02.878492 9042 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-be133eef91634be592094b95357ee200 received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:02.879030 9042 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-be133eef91634be592094b95357ee200 on tablet be133eef91634be592094b95357ee200 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:02.897670 9898 raft_consensus.cc:1217] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:03.052115 9898 raft_consensus.cc:1217] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:03.207295 10036 raft_consensus.cc:1064] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:03.209977 10036 consensus_queue.cc:237] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.217295 9898 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.217417 8906 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.217403 9017 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.219420 10036 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.222466 10100 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.222900 10147 raft_consensus.cc:1064] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:03.225127 10146 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.225835 10147 consensus_queue.cc:237] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.246141 9898 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.248188 10128 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.249465 8778 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.254626 10131 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.253846 10100 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.285492 8628 catalog_manager.cc:5697] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:03.288054 8902 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.288966 9017 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.292975 8906 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.306488 10128 raft_consensus.cc:1064] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:03.312254 10128 consensus_queue.cc:237] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.314289 10131 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.319830 8778 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.341717 9897 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.354784 9898 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.369493 8778 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.364280 9893 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.370864 10054 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.373782 10012 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.375672 10012 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:03.387036 8906 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.392087 10054 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:03.404775 8632 catalog_manager.cc:5697] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:03.438887 8906 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.465039 10012 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.476996 10054 raft_consensus.cc:1064] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:03.480461 9893 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.482604 10054 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.482642 8778 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.491775 8783 consensus_queue.cc:237] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.496482 8910 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.505851 9898 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.507493 10146 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.513664 8906 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.515208 10148 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:03.522006 9898 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.525738 8778 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:03.527720 10148 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:03.534260 8913 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.522356 8630 catalog_manager.cc:5697] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.539072 10036 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.552253 9897 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.557619 10148 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260807 19:49:03.578248 10144 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.589776 8629 catalog_manager.cc:5697] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.602919 9017 consensus_queue.cc:237] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.591589 8913 raft_consensus.cc:2955] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.607560 8996 tablet_service.cc:1558] Processing DeleteTablet for tablet ab8e617d098d425cbddc5d78da43ab99 with delete_type TABLET_DATA_TOMBSTONED (TS a38f4d6088cf4c8cb510200ca0555801 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52256
I20260807 19:49:03.619364 8613 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab8e617d098d425cbddc5d78da43ab99 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:03.623080 9897 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.629269 10148 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.631266 8913 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.635383 10147 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.640435 10147 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.648718 10012 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.648916 9898 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.659646 9897 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.665518 8913 raft_consensus.cc:2955] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.670274 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb33dc9f5a49409bb8817073ec1143da with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260807 19:49:03.670794 10176 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260807 19:49:03.682001 8783 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.648695 10180 tablet_replica.cc:333] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: stopping tablet replica
I20260807 19:49:03.688766 10180 raft_consensus.cc:2243] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:03.687181 8913 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.702767 8630 catalog_manager.cc:5697] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:03.707285 10180 raft_consensus.cc:2272] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:03.722361 8632 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.732852 10180 ts_tablet_manager.cc:1916] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:03.734107 8769 tablet_service.cc:1558] Processing DeleteTablet for tablet fb33dc9f5a49409bb8817073ec1143da with delete_type TABLET_DATA_TOMBSTONED (TS 8e9ad2b5e2a94d4faff90af735c6d792 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60644
I20260807 19:49:03.757787 10180 ts_tablet_manager.cc:1929] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:03.758477 10180 log.cc:1199] T ab8e617d098d425cbddc5d78da43ab99 P a38f4d6088cf4c8cb510200ca0555801: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root/wals/ab8e617d098d425cbddc5d78da43ab99
I20260807 19:49:03.760573 8616 catalog_manager.cc:5028] TS a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): tablet ab8e617d098d425cbddc5d78da43ab99 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:03.764142 10182 tablet_replica.cc:333] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: stopping tablet replica
I20260807 19:49:03.765842 10182 raft_consensus.cc:2243] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:03.767334 10182 raft_consensus.cc:2272] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:03.770731 10182 ts_tablet_manager.cc:1916] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:03.783180 9021 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.796151 9897 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.801437 10182 ts_tablet_manager.cc:1929] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:03.802178 10182 log.cc:1199] T fb33dc9f5a49409bb8817073ec1143da P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/wals/fb33dc9f5a49409bb8817073ec1143da
I20260807 19:49:03.802237 8778 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.803339 10128 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.805868 8613 catalog_manager.cc:5028] TS 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): tablet fb33dc9f5a49409bb8817073ec1143da (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:03.806790 10012 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.816717 10128 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.831674 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be133eef91634be592094b95357ee200 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:03.840975 9017 consensus_queue.cc:237] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:03.826860 9897 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.850494 9898 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.833168 8783 raft_consensus.cc:2955] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.852185 10147 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260807 19:49:03.855736 8910 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:03.861516 10148 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.876210 8629 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:03.880556 10131 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:03.889715 9898 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.895071 8893 tablet_service.cc:1558] Processing DeleteTablet for tablet be133eef91634be592094b95357ee200 with delete_type TABLET_DATA_TOMBSTONED (TS 671d8754170646eaac5748745eba58b4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39358
I20260807 19:49:03.910579 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 413683c7378447239874762d812d75e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:03.910252 10183 tablet_replica.cc:333] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: stopping tablet replica
I20260807 19:49:03.912786 10183 raft_consensus.cc:2243] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:03.913762 10183 raft_consensus.cc:2272] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:03.920009 8910 raft_consensus.cc:2955] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.922894 8630 catalog_manager.cc:5697] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:03.926591 10183 ts_tablet_manager.cc:1916] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:03.942335 8769 tablet_service.cc:1558] Processing DeleteTablet for tablet 413683c7378447239874762d812d75e5 with delete_type TABLET_DATA_TOMBSTONED (TS 8e9ad2b5e2a94d4faff90af735c6d792 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60644
I20260807 19:49:03.944161 10182 tablet_replica.cc:333] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: stopping tablet replica
I20260807 19:49:03.945313 10182 raft_consensus.cc:2243] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:03.946192 10182 raft_consensus.cc:2272] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:03.950273 10182 ts_tablet_manager.cc:1916] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:03.958153 10183 ts_tablet_manager.cc:1929] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:03.958822 10183 log.cc:1199] T be133eef91634be592094b95357ee200 P 671d8754170646eaac5748745eba58b4: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root/wals/be133eef91634be592094b95357ee200
I20260807 19:49:03.960843 8616 catalog_manager.cc:5028] TS 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): tablet be133eef91634be592094b95357ee200 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:03.968273 10182 ts_tablet_manager.cc:1929] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:03.968952 10182 log.cc:1199] T 413683c7378447239874762d812d75e5 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/wals/413683c7378447239874762d812d75e5
I20260807 19:49:03.972121 8613 catalog_manager.cc:5028] TS 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): tablet 413683c7378447239874762d812d75e5 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:05.013170 9017 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:05.019369 8910 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 19:49:05.019784 8970 consensus_peers.cc:597] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fbfe42e0961647279ca7973da14c62a0. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:05.020929 8783 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.021225 10012 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:05.022228 10128 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:05.030018 10148 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.035250 8783 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.035609 8910 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.057823 8629 catalog_manager.cc:5697] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.066995 8910 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:05.089874 9017 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 19:49:05.090709 8853 consensus_peers.cc:597] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 133c01ab82cf4ef3b89a9649b0abf969. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:05.092474 10145 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:05.121773 8783 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.123468 10145 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:05.125988 10145 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.127648 9017 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.149788 8779 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:05.155283 8628 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:05.157742 8783 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
W20260807 19:49:05.165861 8737 consensus_peers.cc:597] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e8d547d81964989bb6cd1b677c971c4. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:05.167321 8913 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.169533 10036 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:05.180666 9017 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.183629 10036 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:05.200578 10146 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.204258 8913 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.210428 9017 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.217273 9021 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:05.226536 8628 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:05.231034 8779 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.232095 8913 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:05.233034 10012 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:49:05.233966 8970 consensus_peers.cc:597] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e005b1fcb3134fa9a076533e372b80f9. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:05.236105 10012 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:05.243602 10128 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.250948 8779 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:05.270355 8629 catalog_manager.cc:5697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:05.270579 8906 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:06.224315 10229 ts_tablet_manager.cc:933] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
I20260807 19:49:06.243117 10229 tablet_copy_client.cc:323] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.193:33185
I20260807 19:49:06.261133 10242 ts_tablet_manager.cc:933] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:06.271912 10242 tablet_copy_client.cc:323] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.194:37659
I20260807 19:49:06.274235 8928 tablet_copy_service.cc:140] P 671d8754170646eaac5748745eba58b4: Received BeginTabletCopySession request for tablet 133c01ab82cf4ef3b89a9649b0abf969 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53310)
I20260807 19:49:06.274809 8928 tablet_copy_service.cc:161] P 671d8754170646eaac5748745eba58b4: Beginning new tablet copy session on tablet 133c01ab82cf4ef3b89a9649b0abf969 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53310: session id = 39975243456d4452b4a4870ac7ebafc5-133c01ab82cf4ef3b89a9649b0abf969
I20260807 19:49:06.278254 10248 ts_tablet_manager.cc:933] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:06.280082 10248 tablet_copy_client.cc:323] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
I20260807 19:49:06.383358 8805 tablet_copy_service.cc:140] P 8e9ad2b5e2a94d4faff90af735c6d792: Received BeginTabletCopySession request for tablet 7e8d547d81964989bb6cd1b677c971c4 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53512)
I20260807 19:49:06.383976 8805 tablet_copy_service.cc:161] P 8e9ad2b5e2a94d4faff90af735c6d792: Beginning new tablet copy session on tablet 7e8d547d81964989bb6cd1b677c971c4 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53512: session id = 39975243456d4452b4a4870ac7ebafc5-7e8d547d81964989bb6cd1b677c971c4
I20260807 19:49:06.386426 10241 ts_tablet_manager.cc:933] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:06.393260 8805 tablet_copy_source_session.cc:215] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:06.402480 10229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8d547d81964989bb6cd1b677c971c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:06.409942 10241 tablet_copy_client.cc:323] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
I20260807 19:49:06.425563 8928 tablet_copy_source_session.cc:215] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:06.429864 10242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 133c01ab82cf4ef3b89a9649b0abf969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:06.431666 10229 tablet_copy_client.cc:806] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:06.432318 10229 tablet_copy_client.cc:670] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
W20260807 19:49:06.435577 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 61304.8us
I20260807 19:49:06.437191 10229 tablet_copy_client.cc:538] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:06.438263 10251 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:06.438995 9042 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet e005b1fcb3134fa9a076533e372b80f9 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:06.439409 9044 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet fbfe42e0961647279ca7973da14c62a0 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:06.439576 9042 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet e005b1fcb3134fa9a076533e372b80f9 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-e005b1fcb3134fa9a076533e372b80f9
I20260807 19:49:06.440599 9044 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet fbfe42e0961647279ca7973da14c62a0 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-fbfe42e0961647279ca7973da14c62a0
I20260807 19:49:06.438898 10251 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:06.443308 10251 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:06.448846 10229 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:06.453830 9044 tablet_copy_source_session.cc:215] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:06.470332 9042 tablet_copy_source_session.cc:215] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:06.472062 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:06.477155 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:06.479938 10248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e005b1fcb3134fa9a076533e372b80f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:06.487502 10241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbfe42e0961647279ca7973da14c62a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:06.488018 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:06.489037 10237 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:06.489876 10242 tablet_copy_client.cc:806] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:06.490592 10242 tablet_copy_client.cc:670] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:06.495350 10242 tablet_copy_client.cc:538] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:06.505642 10242 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:06.522356 10248 tablet_copy_client.cc:806] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:06.523123 10248 tablet_copy_client.cc:670] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:06.527626 10248 tablet_copy_client.cc:538] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:06.532732 10229 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:06.537184 10229 tablet_bootstrap.cc:492] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:06.539561 10229 ts_tablet_manager.cc:1403] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.091s user 0.023s sys 0.003s
I20260807 19:49:06.544682 10229 raft_consensus.cc:359] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.541123 10241 tablet_copy_client.cc:806] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:06.547390 10241 tablet_copy_client.cc:670] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:06.546106 10248 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:06.552449 10241 tablet_copy_client.cc:538] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:06.555464 10242 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:06.556708 10242 tablet_bootstrap.cc:492] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:06.557518 10242 ts_tablet_manager.cc:1403] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.052s user 0.016s sys 0.021s
I20260807 19:49:06.558712 10229 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:06.560521 10229 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.564033 10242 raft_consensus.cc:359] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.565317 10242 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:06.566112 10242 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.573554 10229 ts_tablet_manager.cc:1434] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.032s user 0.004s sys 0.008s
W20260807 19:49:06.575647 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800045880 after lost signal to thread 6318
W20260807 19:49:06.576871 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800045f60 after lost signal to thread 8695
I20260807 19:49:06.576982 10241 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
W20260807 19:49:06.578420 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800101000 after lost signal to thread 8828
I20260807 19:49:06.579115 8805 tablet_copy_service.cc:342] P 8e9ad2b5e2a94d4faff90af735c6d792: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-7e8d547d81964989bb6cd1b677c971c4 received from {username='slave'} at 127.0.0.1:53512
I20260807 19:49:06.579605 8805 tablet_copy_service.cc:434] P 8e9ad2b5e2a94d4faff90af735c6d792: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-7e8d547d81964989bb6cd1b677c971c4 on tablet 7e8d547d81964989bb6cd1b677c971c4 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:06.582094 10242 ts_tablet_manager.cc:1434] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.024s user 0.006s sys 0.005s
W20260807 19:49:06.579655 8689 debug-util.cc:398] Leaking SignalData structure 0x7b08002f9c80 after lost signal to thread 8951
W20260807 19:49:06.590723 8689 debug-util.cc:398] Leaking SignalData structure 0x7b08002ed820 after lost signal to thread 9068
W20260807 19:49:06.592226 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800149800 after lost signal to thread 9937
I20260807 19:49:06.593506 8928 tablet_copy_service.cc:342] P 671d8754170646eaac5748745eba58b4: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-133c01ab82cf4ef3b89a9649b0abf969 received from {username='slave'} at 127.0.0.1:53310
I20260807 19:49:06.594066 8928 tablet_copy_service.cc:434] P 671d8754170646eaac5748745eba58b4: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-133c01ab82cf4ef3b89a9649b0abf969 on tablet 133c01ab82cf4ef3b89a9649b0abf969 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:06.614833 10248 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:06.617430 10248 tablet_bootstrap.cc:492] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:06.618309 10248 ts_tablet_manager.cc:1403] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.072s user 0.021s sys 0.004s
I20260807 19:49:06.621636 10248 raft_consensus.cc:359] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.629451 10248 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:06.630456 10248 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.633490 10248 ts_tablet_manager.cc:1434] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260807 19:49:06.639334 9897 raft_consensus.cc:1217] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:06.646293 9044 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-e005b1fcb3134fa9a076533e372b80f9 received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:06.646823 9044 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-e005b1fcb3134fa9a076533e372b80f9 on tablet e005b1fcb3134fa9a076533e372b80f9 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:06.678640 10241 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:49:06.689194 10241 tablet_bootstrap.cc:492] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:06.690800 10241 ts_tablet_manager.cc:1403] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.114s user 0.038s sys 0.041s
I20260807 19:49:06.694411 10241 raft_consensus.cc:359] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.695397 10241 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:06.696149 10241 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:06.698908 10241 ts_tablet_manager.cc:1434] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260807 19:49:06.701485 9044 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-fbfe42e0961647279ca7973da14c62a0 received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:06.701954 9044 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-fbfe42e0961647279ca7973da14c62a0 on tablet fbfe42e0961647279ca7973da14c62a0 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:06.752099 9897 raft_consensus.cc:1217] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:06.827942 9897 raft_consensus.cc:1217] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:06.893137 9897 raft_consensus.cc:1217] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:07.158654 10145 raft_consensus.cc:1064] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:07.161974 10145 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.194295 9897 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.196384 10145 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:07.205276 9015 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.210922 10221 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:07.243213 8783 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.251669 10219 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.258787 10145 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:07.273528 9017 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.277679 9892 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.287758 8630 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:07.341137 10204 raft_consensus.cc:1064] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:07.344270 10204 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.365417 10237 raft_consensus.cc:1064] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
W20260807 19:49:07.374749 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 46876.5us
I20260807 19:49:07.388921 10237 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.403290 10244 raft_consensus.cc:1064] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:07.412959 8783 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.406260 10244 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.439399 9892 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.444455 10223 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:07.454473 9893 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.457077 10209 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:07.459357 9015 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.463114 9897 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.465348 10238 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:07.468286 8902 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.474006 10204 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:07.475382 10279 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:07.485683 8906 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.487762 8783 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.489166 10223 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:07.489629 8906 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.490437 10128 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:07.491657 10238 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:07.492780 8783 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:07.511438 10128 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260807 19:49:07.528522 10279 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.537917 8902 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.544486 9015 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.547487 8902 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.557087 9897 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.559751 9017 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.564462 10220 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260807 19:49:07.574689 10219 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:07.594771 10238 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.596990 8787 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.604295 9892 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.587492 10220 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.615778 9893 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.621384 10295 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.628327 9897 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.630750 8629 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.641640 8628 catalog_manager.cc:5697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:07.682305 9898 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.691387 8780 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.704012 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 133c01ab82cf4ef3b89a9649b0abf969 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:07.716670 9021 raft_consensus.cc:2955] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.722527 8906 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.750006 8787 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.794205 9892 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.797879 10222 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:07.805199 8630 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 4, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New cstate: current_term: 1 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.810156 8629 catalog_manager.cc:5697] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.815809 9021 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:07.816279 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 85121.4us
I20260807 19:49:07.815624 10269 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.817878 9892 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.829082 9898 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.828097 8913 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.836642 8780 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.844513 8901 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.847707 8769 tablet_service.cc:1558] Processing DeleteTablet for tablet 133c01ab82cf4ef3b89a9649b0abf969 with delete_type TABLET_DATA_TOMBSTONED (TS 8e9ad2b5e2a94d4faff90af735c6d792 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60644
I20260807 19:49:07.848469 10223 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260807 19:49:07.864025 10128 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:07.884476 8628 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New cstate: current_term: 1 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.894734 9892 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.897424 10209 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:07.901309 10128 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.902581 8913 raft_consensus.cc:2955] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.919198 8613 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7e8d547d81964989bb6cd1b677c971c4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:07.890918 9015 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:07.889269 10304 tablet_replica.cc:333] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: stopping tablet replica
I20260807 19:49:07.925177 8787 raft_consensus.cc:2955] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.930927 8996 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e8d547d81964989bb6cd1b677c971c4 with delete_type TABLET_DATA_TOMBSTONED (TS a38f4d6088cf4c8cb510200ca0555801 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:52256
I20260807 19:49:07.937640 10304 raft_consensus.cc:2243] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:07.938989 10304 raft_consensus.cc:2272] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:07.971006 8628 catalog_manager.cc:5697] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:07.987010 8893 tablet_service.cc:1558] Processing DeleteTablet for tablet e005b1fcb3134fa9a076533e372b80f9 with delete_type TABLET_DATA_TOMBSTONED (TS 671d8754170646eaac5748745eba58b4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39358
I20260807 19:49:07.992193 9892 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.993016 8783 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260807 19:49:07.994210 10244 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:07.996394 10237 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:07.986959 10304 ts_tablet_manager.cc:1916] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:08.000372 10305 tablet_replica.cc:333] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: stopping tablet replica
I20260807 19:49:08.022179 8783 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:08.027806 10305 raft_consensus.cc:2243] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:08.017006 9892 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:08.021273 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e005b1fcb3134fa9a076533e372b80f9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:08.014045 10244 raft_consensus.cc:2955] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:08.041941 10310 tablet_replica.cc:333] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: stopping tablet replica
I20260807 19:49:08.056246 10310 raft_consensus.cc:2243] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:08.057138 10310 raft_consensus.cc:2272] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:08.044255 8629 catalog_manager.cc:5697] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 3 to 4, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:08.062836 10310 ts_tablet_manager.cc:1916] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:08.078007 8893 tablet_service.cc:1558] Processing DeleteTablet for tablet fbfe42e0961647279ca7973da14c62a0 with delete_type TABLET_DATA_TOMBSTONED (TS 671d8754170646eaac5748745eba58b4 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:39358
I20260807 19:49:08.094376 10305 raft_consensus.cc:2272] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:08.097982 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fbfe42e0961647279ca7973da14c62a0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:08.117137 10305 ts_tablet_manager.cc:1916] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:08.141254 10304 ts_tablet_manager.cc:1929] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:08.143322 10304 log.cc:1199] T 133c01ab82cf4ef3b89a9649b0abf969 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/wals/133c01ab82cf4ef3b89a9649b0abf969
I20260807 19:49:08.145660 8613 catalog_manager.cc:5028] TS 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): tablet 133c01ab82cf4ef3b89a9649b0abf969 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:08.155368 10310 ts_tablet_manager.cc:1929] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:08.156004 10310 log.cc:1199] T e005b1fcb3134fa9a076533e372b80f9 P 671d8754170646eaac5748745eba58b4: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root/wals/e005b1fcb3134fa9a076533e372b80f9
I20260807 19:49:08.159956 10310 tablet_replica.cc:333] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: stopping tablet replica
I20260807 19:49:08.159878 8616 catalog_manager.cc:5028] TS 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): tablet e005b1fcb3134fa9a076533e372b80f9 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:08.162127 10310 raft_consensus.cc:2243] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:49:08.162950 10310 raft_consensus.cc:2272] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:49:08.167526 10310 ts_tablet_manager.cc:1916] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260807 19:49:08.178817 10305 ts_tablet_manager.cc:1929] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:08.179363 10305 log.cc:1199] T 7e8d547d81964989bb6cd1b677c971c4 P a38f4d6088cf4c8cb510200ca0555801: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-2-root/wals/7e8d547d81964989bb6cd1b677c971c4
I20260807 19:49:08.181272 8616 catalog_manager.cc:5028] TS a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): tablet 7e8d547d81964989bb6cd1b677c971c4 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:08.184803 10310 ts_tablet_manager.cc:1929] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260807 19:49:08.185230 10310 log.cc:1199] T fbfe42e0961647279ca7973da14c62a0 P 671d8754170646eaac5748745eba58b4: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-1-root/wals/fbfe42e0961647279ca7973da14c62a0
I20260807 19:49:08.186548 8616 catalog_manager.cc:5028] TS 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): tablet fbfe42e0961647279ca7973da14c62a0 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:49:09.160640 9015 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
W20260807 19:49:09.186107 8970 consensus_peers.cc:597] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 300f918f185249e6895ab57edd060d31. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:09.203981 8901 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:09.206799 8787 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:49:09.207839 10314 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:10.154531 10319 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.155253 10319 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.157682 10319 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.357784 10319 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.358435 10319 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.517642 10322 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.518316 10319 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.518914 10319 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.519580 10323 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.519457 10319 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.520237 10323 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.521068 10321 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.524019 10321 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.528668 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:10.523073 10323 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.531106 10323 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.523478 10328 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.526376 10321 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.533422 10328 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.537309 10328 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.537914 10328 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.551950 10321 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.552778 10321 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.557489 10321 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.518356 10322 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.559010 10325 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.559428 10321 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.559563 10325 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.560947 10322 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.561484 10322 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.561883 10325 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.560400 10321 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.561964 10322 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.562529 10325 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.563014 10325 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.531698 10323 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.556545 10327 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.564801 10327 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.558301 10319 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.568317 10332 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.569145 10332 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.610620 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:10.564079 10322 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:10.616079 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 45809us
I20260807 19:49:10.618793 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:10.623065 10332 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.624902 10325 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.626160 10331 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.626542 10331 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.628707 10327 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:10.619138 8823 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf0c0 after lost signal to thread 6318
I20260807 19:49:10.629794 10321 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.615527 10314 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.550630 10328 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.618996 10330 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.638577 10314 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.643959 10314 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.626830 10325 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.644958 10330 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.648396 10330 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.649008 10330 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.649482 10330 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.780791 10334 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.781569 10314 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.639344 10223 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.784096 10223 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.779908 10325 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.785614 10314 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.786347 10223 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.787638 10314 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.781433 10337 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.782998 10321 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.781435 10334 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.790588 10321 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.804970 10334 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.800006 10332 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.800763 10222 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.801471 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:10.801367 10322 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.805325 10321 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.790352 10337 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.807807 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:10.813498 10347 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.790216 10331 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.815285 10322 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.816816 10334 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.824307 10334 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.825752 10322 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.826366 10334 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:10.806088 10332 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.821249 10331 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:10.830397 8823 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2420 after lost signal to thread 8695
I20260807 19:49:10.818715 10222 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.838274 10334 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.838567 10336 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.839733 10341 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.829098 10314 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.830631 10223 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.845541 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:10.842047 8823 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2340 after lost signal to thread 8714
I20260807 19:49:10.849738 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:10.850502 9018 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:10.856854 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:10.851038 10332 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.872371 10342 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.873039 10348 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.845091 10331 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.847234 10334 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.849751 10223 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.852559 10314 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.852898 10336 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.870564 10222 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:10.887137 8823 debug-util.cc:398] Leaking SignalData structure 0x7b08002e85c0 after lost signal to thread 8828
I20260807 19:49:10.895871 9015 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:10.830295 10347 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.908859 10358 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.898667 10330 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:10.909792 10331 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.913851 10244 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:10.891170 10342 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.921480 10342 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.922122 10342 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.922624 10342 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.924603 10345 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.925259 10345 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.906390 10222 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.861258 10341 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.927553 10244 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.931831 10244 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.953271 10244 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:10.891798 10348 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.917299 10330 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.955405 10330 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.917445 10331 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.927690 10339 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.960209 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:10.962265 8913 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:10.963941 10352 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.964268 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:10.909516 10358 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.930688 10343 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.932313 10340 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:10.964717 10330 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.932053 10222 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.964586 10352 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:10.964993 8901 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:10.969668 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.028331 8913 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:10.965104 10345 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.958662 10351 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.020592 10334 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:11.031903 10334 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.023209 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:11.015864 10314 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.963452 10331 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:10.968806 10342 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.981628 10223 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.032510 10323 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.982435 10337 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:10.984078 10347 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.044956 10314 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.025876 10359 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.048771 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.027036 10341 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.037809 10332 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.051587 10347 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:10.982005 10343 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.053957 10314 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.043913 10340 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.032493 10334 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.046809 10353 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.046654 10348 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:10.969636 10339 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.056700 10332 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:11.068624 8823 debug-util.cc:398] Leaking SignalData structure 0x7b0800338180 after lost signal to thread 8951
I20260807 19:49:11.068522 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.069677 9850 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:11.070084 10314 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.045409 10360 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.060142 10347 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.046748 10351 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.075493 10339 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.075735 10340 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.078269 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.082582 8913 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.058192 10342 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.087162 10351 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.087452 10342 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.088409 10328 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:11.074074 10332 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.085299 10343 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.085578 10334 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.080019 10360 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:11.090835 8823 debug-util.cc:398] Leaking SignalData structure 0x7b08003157c0 after lost signal to thread 9068
I20260807 19:49:11.086064 10222 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.089046 10323 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.079211 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.072484 10359 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.096407 10352 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.081444 8903 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.108774 10325 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.109191 10360 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.111436 10347 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.109030 9850 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:11.093809 10323 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.080574 10353 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:11.155886 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 195447us
I20260807 19:49:11.120025 10322 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 19:49:11.163599 8737 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s user 0.009s sys 0.012s
W20260807 19:49:11.171123 8737 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s user 0.009s sys 0.018s
W20260807 19:49:11.171583 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 554031us
I20260807 19:49:11.174662 10322 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.175276 10358 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.120187 10323 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:11.123929 8823 debug-util.cc:398] Leaking SignalData structure 0x7b0800074dc0 after lost signal to thread 9937
I20260807 19:49:11.175290 10322 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.188505 10331 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:11.119796 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:11.189409 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 69693.1us
I20260807 19:49:11.190181 9850 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:11.164059 10342 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.190888 10352 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.120404 10336 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.195201 10359 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.193846 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:11.182881 10222 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.194172 10321 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.199002 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:11.192090 10325 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.204506 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.201304 10347 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.225225 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.218201 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.216518 8787 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.235381 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.236697 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.237332 8778 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.236061 8779 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.239581 8780 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.196704 10340 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.203482 10321 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.225550 10327 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.207510 9893 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.225332 10353 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.247849 10321 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.253445 10244 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.205861 10222 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.254118 10244 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.254999 10311 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.249627 10359 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.239718 10347 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.235898 10325 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.251116 10327 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.259243 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.261826 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.263365 10244 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.249794 10340 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.264436 10350 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.258847 10334 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.272445 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.262892 10358 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.205338 10352 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.266005 9017 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.271358 10371 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.272323 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.259958 10336 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.257898 10353 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.273406 9021 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:11.272296 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.268468 10327 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.264189 9838 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:11.273470 10222 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.264297 10363 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.282188 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.287737 10312 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.288587 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.288292 10312 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.289714 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.278992 10342 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.268260 10359 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.293154 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:11.289224 10373 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.276057 10334 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.277151 10358 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.283550 10351 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.265829 10311 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.285636 10353 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.285471 10322 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:11.285035 10336 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.278385 10350 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.284075 10371 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.463672 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.483402 8789 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.483572 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.443238 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.489758 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.492964 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.494505 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.495467 8780 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.302341 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.302536 10363 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.291832 10339 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.336974 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.291791 10360 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.341614 10335 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.499511 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.292488 10222 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.290542 10362 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.508989 9015 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.295595 10347 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.517217 9898 raft_consensus.cc:1240] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:11.515398 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.518042 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.519708 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.521072 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.421259 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.446336 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.336786 10331 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:11.336764 10319 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.446594 10354 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.337854 10328 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.519055 8737 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:11.414547 10345 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.434160 10340 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.421736 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:11.490566 10325 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.297684 10342 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.481710 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:11.569144 9889 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.413664 10221 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.532730 9020 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.497967 10348 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.296501 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:11.341002 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.543866 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.547760 10352 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.550920 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.539633 8790 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.397068 10332 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:11.553781 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.590358 9021 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.555022 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.447849 10375 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.491981 10351 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.570431 10221 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.575779 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.542057 10331 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.602587 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.602779 10331 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:11.603330 8790 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.522609 10339 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.562258 10347 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.540685 10350 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.576653 10348 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.611891 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:11.534617 10360 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.613206 10371 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.556625 10334 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.584424 8778 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.586071 10363 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.586321 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.626116 9015 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.589390 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.591408 9023 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.545571 9018 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.592773 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.551853 9019 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.562574 8788 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.538416 10222 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.587309 9893 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.556466 10362 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.577898 10359 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.562425 10328 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.609244 10347 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.593827 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.572779 10345 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.621194 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.657135 9021 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.514668 10373 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.571902 10319 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.545764 10335 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.644861 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.632395 9889 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:11.593390 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.604059 10329 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:11.645738 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 396717us
I20260807 19:49:11.650933 10347 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.662321 10322 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.656726 10336 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.681782 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.683909 8780 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.602617 10375 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.686386 10332 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:11.650162 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 460047us
I20260807 19:49:11.657953 10312 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.582986 10354 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.687238 10332 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.686465 10322 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.689014 9020 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.678686 10358 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.702250 8970 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:11.688789 10353 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.707588 10322 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.714603 8970 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:11.704962 10332 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.700747 10311 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.695043 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:11.705579 10325 consensus_queue.cc:1059] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260807 19:49:11.720422 10322 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.688449 10329 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.755041 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:11.755539 10329 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.718658 10363 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.731734 10332 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.724881 10358 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.756013 8901 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.725185 10360 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.781392 10363 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:11.800990 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.758203 10348 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.751987 10321 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:11.805794 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.813150 8780 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.759367 10351 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.782882 10373 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:11.726017 10312 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260807 19:49:11.791759 10358 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:11.779937 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 96224.3us
I20260807 19:49:11.802938 10363 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.820994 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.821621 9021 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.798744 10342 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.803732 10375 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.823400 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.825933 8780 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.827180 10380 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:11.831612 10380 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.832013 10380 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.819211 10358 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.818984 10345 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.824124 9888 raft_consensus.cc:1240] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
W20260807 19:49:11.826560 10323 consensus_meta.cc:287] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.632s user 0.009s sys 0.009s
I20260807 19:49:11.825292 10221 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.828686 10360 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.822748 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:11.829366 10321 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.841614 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.886142 9020 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.890561 10372 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.891338 10372 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.892685 8903 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:11.835978 10375 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.838755 10323 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:11.906795 10323 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:11.861349 10327 consensus_meta.cc:287] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.576s user 0.012s sys 0.000s
I20260807 19:49:11.835047 10348 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.841747 10360 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.987610 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.046998 8778 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.922719 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.996675 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.050040 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.051167 8787 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.895694 10345 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.058341 8790 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.974970 10384 ts_tablet_manager.cc:933] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.056483 10319 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.841796 10354 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.991328 10347 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.059830 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.974466 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.060142 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.997164 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.912854 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.920949 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.982590 10311 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:11.984611 10314 consensus_queue.cc:1059] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.147s
I20260807 19:49:11.991807 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.993848 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.911355 10375 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:11.922573 9893 raft_consensus.cc:1240] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.176929 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.177182 10336 raft_consensus.cc:3055] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.177306 8783 raft_consensus.cc:1240] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.054963 10348 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.177683 10336 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:12.178292 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.178735 10336 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.179574 10348 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.037118 9888 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.179868 10336 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.145716 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:11.970204 8901 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:12.183367 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.183753 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.985697 9838 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:12.184489 8908 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.185096 10323 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.185544 10323 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.185535 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.185572 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.187002 8911 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.187096 8908 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.188155 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.862139 9850 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:12.188050 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.186115 10323 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:12.189399 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 351262us
I20260807 19:49:12.189937 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.190694 8911 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:11.994441 10222 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:11.979981 10379 raft_consensus.cc:3055] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:11.896072 10383 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.992085 10372 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.191817 10383 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.179847 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.192538 10321 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.193437 10321 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:12.193846 10321 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.194159 10383 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.038439 10380 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:11.997638 10325 raft_consensus.cc:3055] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:11.929958 10327 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.141273 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.175379 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.193774 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:11.987207 10322 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.203573 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.204398 8908 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.205368 10322 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.196514 8778 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:11.982336 10221 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.206794 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:11.982542 10353 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:11.982766 10359 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:12.207101 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.175751 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:12.175617 10385 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.120s user 0.000s sys 0.001s
I20260807 19:49:12.210973 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.214215 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.211028 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.181937 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.223896 10321 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.200987 10325 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:12.205361 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.222615 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.216194 10379 raft_consensus.cc:740] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:12.223574 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.224649 10380 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.225849 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.063776 10345 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.228726 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.208812 10222 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260807 19:49:12.207755 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.144868 10358 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:12.230758 10331 consensus_meta.cc:287] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.627s user 0.009s sys 0.007s
I20260807 19:49:12.231024 9889 raft_consensus.cc:1240] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.190876 9850 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:12.223253 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.061662 10339 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.143791 8903 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:12.242167 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.232425 10363 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.111598 10328 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.243125 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.237720 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.293466 10319 consensus_queue.cc:1059] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260807 19:49:12.248425 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.300757 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.302125 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.302477 10345 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.302634 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:12.303351 9022 raft_consensus.cc:1240] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.303601 10328 raft_consensus.cc:3055] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.241714 10375 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.304121 10331 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.305653 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.306488 9021 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.308133 10358 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:12.249361 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.294507 10221 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.306706 8779 raft_consensus.cc:2393] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:12.304853 10380 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.258635 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:12.299376 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 108604us
I20260807 19:49:12.312814 8783 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:12.247848 10332 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.314087 9890 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.314077 8908 raft_consensus.cc:1240] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.264523 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:12.233924 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 404961us
I20260807 19:49:12.311968 10380 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.317961 10380 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.317909 9838 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:12.245793 8789 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.319216 10322 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.319845 10322 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.320336 10322 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.320636 10331 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.242373 10379 consensus_queue.cc:260] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.321560 8902 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.295480 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.252504 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.247507 10321 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.321086 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.281876 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.339468 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.340476 8910 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.337956 10363 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:12.325302 8915 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.306295 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.235997 10353 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.304023 10328 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:12.303357 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.246850 10327 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:12.345125 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.345793 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:12.346011 10379 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.343554 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.353991 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.245138 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.360472 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.361389 8902 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.299463 9008 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.249603 10360 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.353338 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.276767 10335 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.311249 10221 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.356899 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:12.350358 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 96553.6us
I20260807 19:49:12.363253 8910 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:12.344611 10347 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:12.367512 8904 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.345719 10314 consensus_queue.cc:1059] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260807 19:49:12.370110 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.244271 10311 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.267168 10323 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.373377 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.365658 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:12.347625 10358 consensus_queue.cc:1059] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260807 19:49:12.364245 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.379947 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.381011 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.244370 10359 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.385049 10314 raft_consensus.cc:3055] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.385682 9891 raft_consensus.cc:1240] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.386238 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.386430 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.377347 9850 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:12.388526 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.388859 9893 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.388916 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.390998 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.421607 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.391775 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.422456 8908 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.423600 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.425092 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.426419 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.427711 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.393584 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:12.429004 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 44451.9us
I20260807 19:49:12.394186 10325 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.395442 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.436976 10323 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.437600 10359 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.377218 10362 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.438078 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.438382 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.397064 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.439627 10358 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.439774 8968 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.440114 10358 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.440642 10358 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.397329 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.442377 10373 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.442845 10373 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.397593 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.422010 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.444830 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:12.398726 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.446200 8968 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.446398 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.446745 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.447028 10334 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:12.447274 8968 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.448546 8968 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.448689 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.449127 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.449395 8968 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.447458 10363 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.450129 10363 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.450234 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.450542 10363 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.400938 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.403085 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.403375 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.404920 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.405048 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.406037 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.456452 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.405905 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.459571 10325 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.408843 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.410986 10314 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:12.389043 10353 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.412532 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.413070 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.413586 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.413816 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.414171 10384 tablet_copy_client.cc:323] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
W20260807 19:49:12.477249 8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.519s (SENT)
I20260807 19:49:12.416483 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.417960 8915 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:12.417853 10322 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.419256 10364 consensus_queue.cc:1059] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260807 19:49:12.483733 9889 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.420104 10311 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.378916 10328 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.428582 10321 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:12.486313 10328 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.467753 10380 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.486361 10364 raft_consensus.cc:3055] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.430779 10332 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.430963 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.431313 9023 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
W20260807 19:49:12.428452 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 55137.4us
I20260807 19:49:12.493196 10311 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:12.494627 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:12.495025 10311 raft_consensus.cc:3055] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.495585 10311 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:12.432104 10347 consensus_queue.cc:1059] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:12.496029 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:12.435425 10390 raft_consensus.cc:3055] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.396910 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:12.498080 10332 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:12.438947 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.443231 10373 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.498183 10390 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:12.498883 10332 raft_consensus.cc:697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:12.438427 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.449415 10334 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.454896 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.448088 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.435801 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.471326 10391 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.480731 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.483395 10322 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:12.475780 8740 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.517s (SENT) [suppressed 5 similar messages]
I20260807 19:49:12.487411 10364 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:12.486874 10321 raft_consensus.cc:697] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:12.484273 10336 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.432082 9888 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.501967 10334 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.498171 8904 raft_consensus.cc:1240] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.508386 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.498019 10347 raft_consensus.cc:3055] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
W20260807 19:49:12.518275 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 68941.6us
I20260807 19:49:12.505848 10323 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election won for term 2
W20260807 19:49:12.524431 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 55561.3us
W20260807 19:49:12.526319 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.509979 9890 raft_consensus.cc:1240] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:12.528007 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.485339 10314 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.510144 10336 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.517310 10334 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.502249 10390 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.511631 10391 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.534905 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:12.519061 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.501101 10345 consensus_queue.cc:1059] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260807 19:49:12.539691 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.541451 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.542104 10319 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.529678 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.543406 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.518610 10347 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:12.545215 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.553673 9850 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:12.544765 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:12.502462 10335 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260807 19:49:12.542718 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.555186 10335 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.555720 10335 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:12.556751 10335 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.550282 10319 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.553522 10347 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.557358 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.536643 10336 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:12.549974 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 81117.6us
I20260807 19:49:12.536281 10391 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.559903 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:12.534181 10323 raft_consensus.cc:697] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:12.543668 10321 consensus_queue.cc:237] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.564702 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.566283 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.566985 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.568727 10222 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.570828 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.558794 10314 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.575594 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.576270 10390 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.560793 10379 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.575186 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.580878 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.562772 10319 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.585991 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:12.570863 10222 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.591979 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.592694 10222 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.571766 10347 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.590060 9044 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet 300f918f185249e6895ab57edd060d31 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:12.585304 10379 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.596083 9044 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet 300f918f185249e6895ab57edd060d31 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-300f918f185249e6895ab57edd060d31
I20260807 19:49:12.596154 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.598055 10345 consensus_queue.cc:1059] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:12.573244 10380 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:12.562004 10397 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:12.534924 10311 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.544090 10364 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.601217 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.581465 10335 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.609652 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.591785 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:12.594188 10362 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.596246 10359 raft_consensus.cc:3055] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.543934 10332 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.596122 10379 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.618096 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:12.594573 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.628736 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:12.616026 10362 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.651613 10362 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.625583 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.611544 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:12.626977 10339 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.624686 10359 raft_consensus.cc:740] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:12.634733 10364 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.638295 10311 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.635545 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:12.637935 10352 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.628233 10392 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260807 19:49:12.659955 10339 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.690373 10373 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.675670 10352 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.701704 8630 catalog_manager.cc:5697] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 2 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:12.634438 10323 consensus_queue.cc:237] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.681799 10327 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.703804 10339 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.710137 9020 raft_consensus.cc:2393] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:12.708534 10352 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.715397 10327 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:12.729988 10327 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.720512 10359 consensus_queue.cc:260] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.773846 10373 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:12.759256 10392 raft_consensus.cc:3055] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:12.801491 10392 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:12.802549 10392 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:12.841358 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 212456us
W20260807 19:49:12.857579 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 313622us
I20260807 19:49:12.859439 8968 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:12.860673 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.863502 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.864229 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:12.865283 8968 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:12.867348 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:12.867981 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.916654 10358 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:12.917573 9850 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:12.921048 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:12.922176 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60044us
I20260807 19:49:12.922312 10321 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 19:49:12.932669 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 84934.5us
I20260807 19:49:12.866488 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.922108 10358 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:12.944864 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:12.962997 10407 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.966627 10407 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:12.967204 10407 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:12.968204 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.940788 10373 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:12.974680 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:12.984634 10325 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.524s user 0.014s sys 0.009s
I20260807 19:49:12.965227 10392 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:13.003799 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:12.975335 10373 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.005259 10373 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:12.965955 10411 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.029466 10359 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.046131 10325 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:13.047088 10325 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:12.970590 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.053910 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:13.055596 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.028460 10411 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.070488 10319 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.059093 10336 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.065745 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:13.076603 10411 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.106972 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.112226 9008 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.115080 8913 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:13.118399 10397 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:13.127900 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:13.129802 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 189728us
I20260807 19:49:13.134174 10408 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.134816 10408 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:13.135223 10408 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.127511 10319 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.162109 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.162647 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.163867 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.165694 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.166018 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:13.167681 10314 consensus_meta.cc:287] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.586s user 0.009s sys 0.017s
I20260807 19:49:13.170351 10373 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.171542 9021 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:13.172972 9020 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
W20260807 19:49:13.174229 10328 consensus_meta.cc:287] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.687s user 0.021s sys 0.041s
W20260807 19:49:13.176630 10222 consensus_meta.cc:287] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.583s user 0.012s sys 0.015s
I20260807 19:49:13.177069 10222 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.178212 10397 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260807 19:49:13.178889 10364 consensus_meta.cc:287] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.515s user 0.021s sys 0.025s
I20260807 19:49:13.179857 10222 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:13.170375 10362 consensus_meta.cc:287] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.518s user 0.008s sys 0.006s
I20260807 19:49:13.187919 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.188024 10362 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.190567 10362 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.191447 10352 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:13.198710 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 333685us
I20260807 19:49:13.200217 8968 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:13.192155 10373 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.203823 10352 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:13.194743 10363 consensus_meta.cc:287] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.744s user 0.013s sys 0.009s
I20260807 19:49:13.213136 10363 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.213692 8849 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.214736 9015 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.215595 10364 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.216068 10364 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.216449 10364 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.211174 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.206172 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.222786 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.223275 10345 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:13.225909 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:13.211823 10339 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.231261 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:13.218403 10363 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.236693 10360 consensus_queue.cc:1059] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:13.239778 8790 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
W20260807 19:49:13.251139 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.249861 10353 consensus_queue.cc:1059] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 19:49:13.254092 8968 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:13.229429 10325 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:13.176194 10327 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.257468 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.261473 8779 raft_consensus.cc:2393] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:13.258016 10336 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.268100 10372 consensus_queue.cc:1059] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260807 19:49:13.286769 10334 consensus_meta.cc:287] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.731s user 0.011s sys 0.019s
W20260807 19:49:13.287629 10347 consensus_meta.cc:287] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.686s user 0.005s sys 0.026s
I20260807 19:49:13.287384 10334 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.289819 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.290076 10408 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.295593 10407 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:13.301363 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.303992 10372 raft_consensus.cc:3055] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.304606 10372 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:13.305161 10311 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.626s user 0.018s sys 0.007s
I20260807 19:49:13.305804 10372 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.307243 10372 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.276785 8968 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:13.276259 10336 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.316700 10336 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.317138 10336 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.317008 10411 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.321449 8915 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:13.324846 9017 raft_consensus.cc:1240] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.325628 9850 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:13.326037 10334 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.326812 10323 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:13.327368 10323 raft_consensus.cc:697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
W20260807 19:49:13.328492 10390 consensus_meta.cc:287] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.681s user 0.013s sys 0.018s
I20260807 19:49:13.328542 10323 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:13.330467 10327 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:13.333577 10335 consensus_meta.cc:287] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.696s user 0.012s sys 0.010s
I20260807 19:49:13.308292 10408 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.337247 10408 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.337895 10408 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.338837 9023 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:13.341599 8903 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:13.372051 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.371433 8780 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
W20260807 19:49:13.374435 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 97751.9us
W20260807 19:49:13.328552 10379 consensus_meta.cc:287] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.679s user 0.018s sys 0.004s
I20260807 19:49:13.379292 9850 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:13.381487 9027 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.381583 8968 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:13.382411 10221 raft_consensus.cc:3055] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.382849 10221 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:13.384298 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.385808 8787 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.386085 8915 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:13.388259 10397 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:13.368139 8901 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
W20260807 19:49:13.385196 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.388232 8780 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.384734 10392 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.369306 10339 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.401758 10345 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:13.384064 10221 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.408399 10221 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.409328 8779 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:13.392727 10321 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260807 19:49:13.416230 8779 raft_consensus.cc:1240] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.424643 8776 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:13.424898 10426 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:13.425602 10426 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.380110 10411 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.374984 10360 consensus_queue.cc:1059] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260807 19:49:13.435396 8789 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:13.437284 10426 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.438062 8901 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:13.443254 8787 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:13.451038 9044 tablet_copy_source_session.cc:215] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:13.393249 10392 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.459676 8787 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.460917 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.391072 10379 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.463274 10332 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260807 19:49:13.483403 10429 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.432320 10334 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.492705 10334 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:13.465842 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 55291.1us
I20260807 19:49:13.489259 10429 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:13.478184 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.496524 9027 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.498834 8778 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.485627 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.479038 10408 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:13.489780 10391 consensus_meta.cc:287] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.818s user 0.021s sys 0.060s
I20260807 19:49:13.487238 10407 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.493294 10334 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.510479 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.511660 10430 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:13.456642 10390 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:13.509199 8789 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.470961 10392 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.490401 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.532902 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.535327 8783 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.536465 10336 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.537889 9015 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.521358 10339 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.530997 10397 raft_consensus.cc:697] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:13.501266 10364 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.543870 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.547024 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.528535 10391 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.549894 10426 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:13.551981 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:13.557068 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 104118us
I20260807 19:49:13.557541 10430 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:13.557933 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.558179 10430 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:13.559520 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.560413 10327 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.560832 10327 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.561232 10327 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.541980 10390 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.561623 10411 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:13.543094 10336 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.543072 10339 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.573493 10411 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.573966 10358 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:13.529071 10429 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.577054 10379 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.570559 10408 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:13.577493 10411 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.572695 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.579375 10353 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:13.579630 8849 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.580917 10390 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.583518 10391 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.584506 10431 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:13.572751 10339 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.583484 8849 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.558190 10359 raft_consensus.cc:3055] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.593283 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.577147 10429 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.596760 10391 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:13.602298 10364 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.602531 10353 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:13.579434 10392 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.603253 10334 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.604081 10327 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.613675 10384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 300f918f185249e6895ab57edd060d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:13.608006 8783 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:13.616277 9017 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.615643 10392 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.595147 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.592445 10359 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:13.618546 8776 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.594398 10430 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.630293 10334 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.635083 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:13.631588 10391 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.639688 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.615509 10327 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.593220 10358 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.624792 10408 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.646394 8780 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.650125 8778 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.654584 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.657229 10314 consensus_queue.cc:1059] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:13.661114 8790 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.643384 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:13.666108 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:13.664794 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:13.647614 10408 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:13.690711 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 56892us
I20260807 19:49:13.660245 10327 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.699402 10408 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.710387 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.655323 10373 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260807 19:49:13.712028 10327 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.718683 10359 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.720044 10359 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.722693 10391 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.714304 10364 consensus_queue.cc:1059] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260807 19:49:13.730876 8790 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:13.731564 10358 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:13.741647 10327 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.665459 10397 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:13.655848 10390 consensus_queue.cc:1059] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260807 19:49:13.758287 10347 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.761711 10347 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:13.765345 10347 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:13.766148 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:13.771246 8776 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:13.773027 10327 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.773583 10327 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.773988 10327 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:13.784811 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 85042.7us
I20260807 19:49:13.787861 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.789027 10347 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.789489 10347 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.789857 10347 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.793367 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.794575 10379 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.795071 10379 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:13.795729 10379 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.699949 8783 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:13.806689 10379 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.811029 10411 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.816951 10336 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.817085 10411 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:13.678964 10334 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
I20260807 19:49:13.817664 10336 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:13.815388 10392 raft_consensus.cc:3055] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.818904 10336 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.819351 10392 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:13.818691 10411 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.820765 10334 raft_consensus.cc:3055] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:13.821856 10392 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.822716 10334 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:13.823608 10392 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.824203 10334 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.825775 10334 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.820223 10336 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.832013 10379 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:13.836606 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.840480 10339 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.842031 8789 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:13.843405 10339 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:13.795035 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:49:13.888233 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.903404 10408 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:13.919489 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 124554us
I20260807 19:49:13.917286 10339 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 19:49:13.919973 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 443093us
W20260807 19:49:13.927392 10323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s user 0.056s sys 0.125s
I20260807 19:49:13.935705 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:13.927799 10339 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:13.935308 10323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s user 0.056s sys 0.125s
I20260807 19:49:13.922974 10347 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:13.940443 10379 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:13.944610 10339 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:13.954375 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 291415us
W20260807 19:49:13.960243 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.961134 8740 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:13.950969 10429 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260807 19:49:13.951316 10379 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:13.980036 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:13.997808 9017 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:13.992952 10379 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:13.999651 10360 consensus_queue.cc:1059] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260807 19:49:14.009465 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:13.971262 10327 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.008417 10336 consensus_queue.cc:1059] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260807 19:49:14.015183 10336 raft_consensus.cc:3055] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.006105 10408 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.015606 10336 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.026703 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.017570 10437 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.019006 10327 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.031710 8968 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.014392 10392 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260807 19:49:14.070931 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:14.071529 8790 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.073239 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.072970 10336 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.074421 10336 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.066617 10347 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.075204 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:14.075861 8789 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.036625 10364 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.077195 10397 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:14.077857 10397 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.080041 10397 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.074680 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.081310 8788 raft_consensus.cc:1240] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.074766 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:14.083271 8776 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:14.065981 10437 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.084054 10411 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.086139 10437 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.087929 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:14.088827 9890 raft_consensus.cc:2393] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:14.089475 10392 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.089885 10392 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.090762 10392 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.091894 10392 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.092921 10407 raft_consensus.cc:3055] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.093334 10407 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.094304 10407 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.092824 10379 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.097854 10379 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.103279 10345 raft_consensus.cc:3055] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.103750 10345 raft_consensus.cc:740] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.104285 8789 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.104763 10345 consensus_queue.cc:260] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.106139 10345 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.108819 10359 raft_consensus.cc:3055] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.109265 10359 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.110208 10359 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.111691 10442 raft_consensus.cc:3055] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.112152 10442 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.113019 10442 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.113969 10442 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.076421 10390 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.118932 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.119938 8901 raft_consensus.cc:2393] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.064471 10408 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.121448 10359 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.026224 10430 consensus_queue.cc:1059] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260807 19:49:14.134392 10442 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.134871 10442 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.136018 10407 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.136016 10442 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.145433 10442 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.153946 10384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:33452 (request call id 90) took 2813 ms. Trace:
I20260807 19:49:14.154358 10384 rpcz_store.cc:276] 0807 19:49:11.341007 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:11.391476 (+ 50469us) service_pool.cc:224] Handling call
0807 19:49:12.080925 (+689449us) ts_tablet_manager.cc:934] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
0807 19:49:13.585215 (+1504290us) tablet_copy_client.cc:368] Tablet copy session begun
0807 19:49:13.898757 (+313542us) tablet_metadata.cc:665] Metadata flushed
0807 19:49:13.898891 (+ 134us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 19:49:14.153190 (+254299us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 19:49:14.153483 (+ 293us) tablet_replica.cc:206] Creating consensus instance
0807 19:49:14.153744 (+ 261us) ts_tablet_manager.cc:989] Replica registered.
0807 19:49:14.153829 (+ 85us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":39620,"tablet-copy.queue_time_us":463958,"thread_start_us":557914,"threads_started":2}
I20260807 19:49:14.155923 10384 tablet_copy_client.cc:806] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:14.159780 8790 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.170043 10339 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.172305 10339 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:14.173655 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:14.185298 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 601952us
I20260807 19:49:14.189421 10426 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:37368 (request call id 5) took 5028 ms. Trace:
I20260807 19:49:14.190059 10426 rpcz_store.cc:276] 0807 19:49:09.158416 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:09.158543 (+ 127us) service_pool.cc:224] Handling call
0807 19:49:09.167905 (+ 9362us) log.cc:844] Serialized 499 byte log entry
0807 19:49:14.185881 (+5017976us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1088,"raft.run_cpu_time_us":1702,"raft.run_wall_time_us":1700,"spinlock_wait_cycles":98816,"thread_start_us":32726,"threads_started":2,"wal-append.queue_time_us":13924,"wal-append.run_cpu_time_us":18301,"wal-append.run_wall_time_us":1184694}
I20260807 19:49:14.192682 10397 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:14.192889 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.193679 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.186875 10384 tablet_copy_client.cc:670] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:14.192740 10426 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.196391 8790 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.198436 10397 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.199831 10390 raft_consensus.cc:697] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.200765 10397 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:14.200963 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.201869 10336 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.202435 10336 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.203467 10430 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.204530 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:14.204715 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.206391 9017 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:14.206737 9015 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.193248 10373 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.207964 10335 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.205528 10430 consensus_queue.cc:1059] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.205469 9020 raft_consensus.cc:2393] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.207566 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.196054 10426 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:14.213572 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 204079us
I20260807 19:49:14.214982 10336 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.215293 10426 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:14.215528 10452 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.217571 8968 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.218432 8968 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.197836 10364 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.209046 10390 consensus_queue.cc:237] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.225832 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.222939 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:14.228790 8915 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:14.220781 10411 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.216113 10452 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.228837 9890 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.230190 9850 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:14.220299 10358 consensus_queue.cc:1059] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260807 19:49:14.220553 10451 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.231128 10411 raft_consensus.cc:697] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.227097 10323 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260807 19:49:14.232154 10411 consensus_queue.cc:237] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.208305 10373 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:14.222251 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.234441 10358 raft_consensus.cc:3055] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.234843 10358 raft_consensus.cc:740] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.235605 10452 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.236929 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.237651 10451 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.239097 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:14.241235 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.237818 8970 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.244827 8783 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.249042 8629 catalog_manager.cc:5697] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 2 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:14.250917 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.251519 10442 consensus_queue.cc:1059] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.250911 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:14.252717 10453 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.234835 10373 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:14.253106 10442 raft_consensus.cc:3055] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.253502 10442 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.254154 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:14.254784 9020 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:14.255255 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:14.256635 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 49239.9us
I20260807 19:49:14.254307 10442 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.257337 10435 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.257791 10435 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.258262 10435 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.258589 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.258946 10359 raft_consensus.cc:3055] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:14.259389 10359 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.235723 10358 consensus_queue.cc:260] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.261075 10358 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.257647 10430 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.262146 10372 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.262651 10430 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.263063 10430 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.263522 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.264439 10335 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.257654 10442 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:14.244441 8972 leader_election.cc:336] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.519s (SENT)
I20260807 19:49:14.266156 8972 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.266907 10430 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:14.260264 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 301736us
I20260807 19:49:14.268668 10336 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.268911 10430 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.271723 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:14.253124 10453 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.272812 10453 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.273274 10314 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.273846 10314 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.274174 10364 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.279086 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:14.279893 9889 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.281327 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 60) took 1576 ms. Trace:
I20260807 19:49:14.283044 8970 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.286871 10339 raft_consensus.cc:2764] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260807 19:49:14.279508 10451 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.290664 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.291326 8779 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.292863 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:14.267479 8628 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New cstate: current_term: 1 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:14.298715 10384 tablet_copy_client.cc:538] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:14.260599 10397 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.278290 10314 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.288750 10345 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.300063 10459 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.309760 10459 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:14.310242 10459 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.289392 10392 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.281683 8630 rpcz_store.cc:276] 0807 19:49:12.695363 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:12.695523 (+ 160us) service_pool.cc:224] Handling call
0807 19:49:14.271709 (+1576186us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:13.096876 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:13.102043 (+ 5167us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:13.102098 (+ 55us) write_op.cc:435] Acquired schema lock
0807 19:49:13.102132 (+ 34us) tablet.cc:662] Decoding operations
0807 19:49:13.102724 (+ 592us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:13.102825 (+ 101us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:13.102871 (+ 46us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:49:13.147427 (+ 44556us) tablet.cc:701] Row locks acquired
0807 19:49:13.150628 (+ 3201us) write_op.cc:260] PREPARE: finished
0807 19:49:13.150852 (+ 224us) write_op.cc:270] Start()
0807 19:49:13.151082 (+ 230us) write_op.cc:276] Timestamp: P: 1786132153150787 usec, L: 0
0807 19:49:13.151118 (+ 36us) op_driver.cc:348] REPLICATION: starting
0807 19:49:13.152167 (+ 1049us) log.cc:844] Serialized 532 byte log entry
0807 19:49:14.000236 (+848069us) op_driver.cc:464] REPLICATION: finished
0807 19:49:14.196678 (+196442us) write_op.cc:301] APPLY: starting
0807 19:49:14.196788 (+ 110us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:14.196940 (+ 152us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:14.197095 (+ 155us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:14.197395 (+ 300us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:14.197661 (+ 266us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:49:14.197726 (+ 65us) write_op.cc:312] APPLY: finished
0807 19:49:14.259399 (+ 61673us) log.cc:844] Serialized 22 byte log entry
0807 19:49:14.261055 (+ 1656us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:14.261134 (+ 79us) write_op.cc:454] Released schema lock
0807 19:49:14.261184 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:14.261624 (+ 440us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:14.261650 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":189223,"apply.run_cpu_time_us":2365,"apply.run_wall_time_us":64913,"num_ops":1,"prepare.queue_time_us":388704,"prepare.run_cpu_time_us":27609,"prepare.run_wall_time_us":148947,"raft.queue_time_us":429171,"raft.run_cpu_time_us":5294,"raft.run_wall_time_us":250485,"replication_time_us":816510,"thread_start_us":982608,"threads_started":4,"wal-append.queue_time_us":243424}]]}
W20260807 19:49:14.317308 8972 leader_election.cc:336] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.600s (SENT)
I20260807 19:49:14.281154 10359 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.326572 10359 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:14.328738 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.305557 10435 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.319094 10453 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.313591 10392 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.339398 8740 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:14.338750 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:14.345539 9889 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.332842 9890 raft_consensus.cc:2393] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:14.349247 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.350178 8901 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.353431 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:14.354189 8779 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:14.355046 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.332322 10435 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.357318 10408 raft_consensus.cc:2764] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260807 19:49:14.357957 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:14.358565 9027 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:14.366197 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 50493.6us
I20260807 19:49:14.370007 8779 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:14.371507 8968 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.372449 10358 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:14.373427 10358 raft_consensus.cc:697] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.369408 10411 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.374722 10358 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.374679 10411 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.380896 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:14.381924 9890 raft_consensus.cc:2393] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:14.384513 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.385363 8779 raft_consensus.cc:2393] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:14.387149 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.388142 10336 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.389402 9027 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:14.389703 9889 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:14.390754 9850 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.391669 10435 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:14.391824 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 52527.1us
I20260807 19:49:14.392534 10435 raft_consensus.cc:697] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:14.394356 10435 consensus_queue.cc:237] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.394855 8740 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.395685 10339 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.397171 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.399979 10336 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.338083 10453 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.361248 10392 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.384582 10411 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.418057 8901 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:14.410678 10339 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.421748 8740 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.423164 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.424068 10408 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.424870 10408 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.426538 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.427328 10339 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.432065 8740 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:14.433027 10408 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.433457 10408 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:14.433866 10408 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.438953 10397 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.443022 10430 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.443792 10430 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.446579 10384 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:14.448589 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:14.449513 10430 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.449680 9027 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:14.451385 8856 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.452379 8740 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.459784 10339 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.452003 10459 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.455154 10430 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.476410 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.464260 10459 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.482600 10339 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.481948 10430 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.483865 10430 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.485051 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.489903 10339 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.491425 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.495270 10339 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.494696 8630 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 2 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260807 19:49:14.497666 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.496980 8630 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 2 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.500977 10392 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.504424 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:14.505031 8779 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.512137 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.514109 8779 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:14.515338 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.501160 8630 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 2 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.524313 8630 catalog_manager.cc:5697] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 3 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.527314 8740 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:14.524183 10459 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:14.522854 10453 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:14.531499 10379 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.535820 9027 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.535820 9020 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.539294 10453 raft_consensus.cc:697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.539999 10359 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.540480 10453 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.541208 10359 raft_consensus.cc:3055] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.541728 10359 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.543144 9027 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.543320 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.543500 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.544119 8903 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.540019 8972 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.546118 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.550441 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.545989 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.553723 9015 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.554390 8908 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.548852 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.551152 8915 raft_consensus.cc:2376] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2. Re-sending same reply.
I20260807 19:49:14.552732 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.552819 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:14.553081 10335 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.553505 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.558919 9889 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.560753 10442 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.561210 10442 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.562155 10408 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.562494 10442 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.563208 8904 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.563696 10442 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.558537 8902 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:14.565352 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.566947 10336 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.568573 8972 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:14.569833 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.571380 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.571985 8915 raft_consensus.cc:2376] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2. Re-sending same reply.
I20260807 19:49:14.572868 10336 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.573251 10345 consensus_queue.cc:1059] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:14.542654 10359 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.573256 10335 consensus_queue.cc:1059] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 19:49:14.576836 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.576014 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.576220 10372 raft_consensus.cc:3055] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.575456 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:14.578214 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.579054 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.546592 10392 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.580468 8911 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.581844 8915 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.564282 10408 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.578589 10372 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.584827 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.585227 10372 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.586535 10372 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.587468 8901 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.590049 8910 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.590492 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.589970 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.591836 10359 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.581377 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.591470 10345 raft_consensus.cc:3055] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:14.594720 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.594746 10345 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.596278 10345 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.597638 10345 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.600587 8906 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.605818 8910 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.607020 8902 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.608083 8915 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.610759 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.623229 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.630084 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.630854 8902 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:14.631062 8629 catalog_manager.cc:5697] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.635819 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.636664 8901 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.638092 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.637580 8629 catalog_manager.cc:5697] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.638850 8901 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.640758 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.641392 8901 raft_consensus.cc:2376] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2. Re-sending same reply.
I20260807 19:49:14.642076 9889 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:14.643371 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.644093 8903 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.644172 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.644814 8911 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:14.645030 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.645609 8903 raft_consensus.cc:2376] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2. Re-sending same reply.
I20260807 19:49:14.646416 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.647109 8903 raft_consensus.cc:2393] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:14.647038 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.647843 8902 raft_consensus.cc:2376] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2. Re-sending same reply.
I20260807 19:49:14.647245 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.648408 8915 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.648391 10397 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.650416 10397 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.655714 10397 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.656718 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.657601 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:14.658289 9015 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:14.658499 8908 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.659647 9850 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.664624 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.665412 8903 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:14.667443 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.667584 10397 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.639599 8629 catalog_manager.cc:5697] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.668622 10397 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.669803 10397 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.679399 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.681023 8779 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:14.682884 10392 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.683110 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.684775 8915 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.686039 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.686724 8901 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.689507 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.689975 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.690397 10392 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.690559 8902 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.691161 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.696309 8910 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:14.696485 8911 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.699055 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.699169 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.699743 8903 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.699815 8910 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.696609 8629 catalog_manager.cc:5697] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:14.701954 8915 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.691071 10435 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260807 19:49:14.708729 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.709764 8906 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.712226 8629 catalog_manager.cc:5697] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:14.714340 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.715718 8915 raft_consensus.cc:2393] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:14.716897 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 64407.5us
I20260807 19:49:14.688875 8904 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:14.713241 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:14.717285 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.718595 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.719221 8906 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.723218 8903 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.689513 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.724457 8910 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.725555 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.730779 8902 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.731093 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.732192 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.734018 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.734536 8915 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.734895 8910 raft_consensus.cc:2393] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.735459 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.733736 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.736260 8906 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.737007 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.737180 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.736192 8787 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:14.737740 8790 raft_consensus.cc:2393] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:14.737900 8906 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.736233 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.739105 10334 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.739503 10430 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.740684 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.740865 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.742131 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.743109 8913 raft_consensus.cc:2368] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:14.744493 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.744910 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.747519 10334 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.748736 10430 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.749972 8972 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.742636 8908 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.752373 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.750005 8903 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.753748 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.759739 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:14.760453 9015 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.760706 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.761608 8789 raft_consensus.cc:2393] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:14.762413 10451 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.762956 10451 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.763388 10451 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.771991 10397 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.773039 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.774417 8902 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:14.775303 9015 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:14.777781 8910 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.781203 8789 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:14.782830 8915 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.760260 10384 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260807 19:49:14.784405 8787 raft_consensus.cc:1275] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:14.790330 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.783550 8906 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.793470 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.788858 10408 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:14.795674 10408 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.796808 10334 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:14.798609 10397 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:14.780066 8856 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:14.799437 8911 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.789952 10384 tablet_bootstrap.cc:492] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:14.800431 10430 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:14.800822 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:14.801404 8911 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:14.802120 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 52209.1us
I20260807 19:49:14.808995 8787 raft_consensus.cc:1240] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.802774 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:14.812265 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:14.810302 8972 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.811643 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.812835 9015 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.813519 8789 raft_consensus.cc:2393] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:14.800870 10384 ts_tablet_manager.cc:1403] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.355s user 0.014s sys 0.017s
I20260807 19:49:14.815526 8911 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:14.816982 9889 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.800963 10430 raft_consensus.cc:697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:14.801607 10408 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.831535 8972 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.802642 10334 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260807 19:49:14.826713 10426 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.836118 8628 catalog_manager.cc:5697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:14.837179 10426 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:14.837616 10426 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.840202 10430 consensus_queue.cc:237] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.847393 8787 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:14.847707 8779 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.849607 10336 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.851116 8972 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:14.852559 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.854724 9015 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:14.855037 10336 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.856175 10336 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.856670 10336 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.855188 10411 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260807 19:49:14.858438 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.859027 8737 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:14.861735 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.862315 10379 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:14.862746 10379 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:14.863083 10379 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:14.845281 9850 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:14.865604 10435 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:14.866060 8910 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:14.866101 10435 raft_consensus.cc:697] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:14.870280 10426 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.876526 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:14.877411 8776 raft_consensus.cc:2393] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:14.867868 10435 consensus_queue.cc:237] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.854130 8972 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.882239 10321 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:14.884231 10452 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.885666 8972 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.893615 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.887269 10426 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:14.834568 8901 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.898015 8972 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.902951 8972 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.904937 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.901990 10452 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.906045 10358 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.906605 10358 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.907572 10358 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.908666 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:14.909452 10392 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.909981 10392 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.897174 10451 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.910923 10392 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.911908 10451 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:14.906045 10452 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.913101 10452 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.905781 10453 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:14.918361 8779 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.918502 8972 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.922510 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:14.925871 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.928108 8972 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.931434 9890 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:14.917483 10453 raft_consensus.cc:697] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:14.903669 10334 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.934976 8972 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.882225 10379 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:14.937175 8779 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.925922 10451 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:14.934053 10334 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.936767 10411 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.943408 10411 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.939199 10453 consensus_queue.cc:237] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.945510 8972 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:14.943845 10411 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.926255 10336 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:14.966796 8789 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.967175 10451 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:14.975003 8790 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:14.944139 10334 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.936167 10426 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260807 19:49:14.982793 9020 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:14.980113 10451 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:14.985551 10451 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:14.989540 10379 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:14.993175 8908 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:14.995772 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:14.996979 9890 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:15.005419 8776 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.005864 8790 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.006809 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.006954 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.007655 8789 raft_consensus.cc:2368] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 2. Current term is 3.
I20260807 19:49:15.008266 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.008661 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.008798 8779 raft_consensus.cc:1240] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.009044 8783 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:15.009344 8787 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:15.009537 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.010145 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.007593 8780 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
W20260807 19:49:15.010958 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:14.960320 10364 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 19:49:15.020020 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.021377 8788 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.022696 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.023422 8776 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 3
W20260807 19:49:15.027643 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 82195.8us
I20260807 19:49:15.028476 8972 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:15.024742 10364 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:15.011953 10451 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:15.050040 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 69561us
I20260807 19:49:15.050707 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:15.051729 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.052183 10336 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.052923 10345 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:15.043918 10364 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.065425 8783 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:15.056486 10384 raft_consensus.cc:359] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.057343 10397 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:15.058218 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.058845 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.060024 10452 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.053442 10345 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.061362 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.062955 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.080060 10452 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.063805 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.081939 10345 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:15.067502 8783 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.083041 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:15.084251 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:15.085037 9893 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:15.080078 10451 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.086354 9890 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:15.087661 10384 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:15.067772 8972 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:15.088507 8737 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.089552 10336 raft_consensus.cc:2764] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260807 19:49:15.066365 8787 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:15.089438 8628 catalog_manager.cc:5697] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 3, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 3 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:15.070650 10426 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:15.071661 8630 catalog_manager.cc:5697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:15.093226 10411 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.095523 10411 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.088079 10334 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.099273 10334 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.099905 8972 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:15.076258 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.087805 8853 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:15.105126 8789 raft_consensus.cc:2376] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2. Re-sending same reply.
I20260807 19:49:15.089564 10379 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:15.106721 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.106727 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.073626 10508 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.112524 8972 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:15.116205 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.119762 10379 raft_consensus.cc:697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:15.088441 10384 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:15.121596 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.126067 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.093068 10426 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:15.114529 10372 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:15.158434 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.114993 10345 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:15.164034 8789 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:15.169881 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.179772 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.180450 8789 raft_consensus.cc:2376] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2. Re-sending same reply.
I20260807 19:49:15.171810 10372 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:15.181751 10372 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:15.172153 10426 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.186125 8776 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:15.171854 10453 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.187021 10358 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:15.187687 10408 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:15.188369 10508 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:15.188223 10408 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.189497 10508 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.190447 10408 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:15.196920 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 84608.9us
I20260807 19:49:15.197841 8972 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:15.120895 10379 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.223696 8780 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:15.190809 10364 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.236949 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.237638 8780 raft_consensus.cc:2376] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2. Re-sending same reply.
I20260807 19:49:15.199163 10408 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:15.239447 8972 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:15.240912 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.200047 10392 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:15.234875 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:15.198868 10345 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.248157 9890 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:15.251683 10364 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.253186 9027 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:15.253481 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.256536 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.232652 10334 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.232288 10512 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:15.263849 8972 leader_election.cc:400] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
W20260807 19:49:15.267751 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.264262 8972 leader_election.cc:403] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260807 19:49:15.262018 10334 raft_consensus.cc:697] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:15.255858 10513 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.270545 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.279101 8789 raft_consensus.cc:2376] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2. Re-sending same reply.
I20260807 19:49:15.284703 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:15.283324 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.256026 10392 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:15.293715 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.266188 10359 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260807 19:49:15.246758 10408 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:15.287627 10513 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:15.290041 10358 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:15.290714 10334 consensus_queue.cc:237] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.326236 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.326929 8908 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 3
W20260807 19:49:15.327288 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 44066.6us
I20260807 19:49:15.330191 10392 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.309398 10430 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.335515 8903 raft_consensus.cc:1275] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:15.338897 8915 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:15.339622 8903 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:15.335812 8910 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:15.333899 10321 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260807 19:49:15.324131 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:15.326869 10408 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:15.346675 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:15.349826 8915 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:15.353181 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.349984 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.355626 10384 ts_tablet_manager.cc:1434] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.532s user 0.011s sys 0.000s
W20260807 19:49:15.356149 10384 ts_tablet_manager.cc:1467] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Tablet startup took 532ms
I20260807 19:49:15.340113 10358 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:15.322714 10530 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260807 19:49:15.360965 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.362535 8902 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:15.365053 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.365798 8915 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.368602 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.370443 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.371210 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.374971 9027 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:15.376765 10513 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:15.377578 10521 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.063s user 0.001s sys 0.000s
I20260807 19:49:15.378886 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:15.384487 10515 log.cc:927] Time spent T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.095s user 0.000s sys 0.001s
I20260807 19:49:15.383764 9023 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:15.387116 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.401598 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.404891 10426 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.407965 9888 raft_consensus.cc:1217] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260807 19:49:15.362566 10358 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.373916 10345 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:15.402025 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:15.410122 10358 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.413818 9027 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:49:15.414247 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.414873 10392 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:15.415117 10372 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.415766 10392 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.418251 8915 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:15.424413 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.407800 10426 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.424474 9021 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:15.426131 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.426800 8902 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.430109 9017 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:15.408954 9888 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:15.444975 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:15.446054 10372 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:15.356524 10384 ts_tablet_manager.cc:1469] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Trace:
0807 19:49:11.341007 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:11.391476 (+ 50469us) service_pool.cc:224] Handling call
0807 19:49:12.080925 (+689449us) ts_tablet_manager.cc:934] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
0807 19:49:13.585215 (+1504290us) tablet_copy_client.cc:368] Tablet copy session begun
0807 19:49:13.898757 (+313542us) tablet_metadata.cc:665] Metadata flushed
0807 19:49:13.898891 (+ 134us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 19:49:14.153190 (+254299us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 19:49:14.153483 (+ 293us) tablet_replica.cc:206] Creating consensus instance
0807 19:49:14.153744 (+ 261us) ts_tablet_manager.cc:989] Replica registered.
0807 19:49:14.153829 (+ 85us) inbound_call.cc:177] Queueing success response
0807 19:49:14.445978 (+292149us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0807 19:49:14.823771 (+377793us) ts_tablet_manager.cc:1435] Starting tablet replica
0807 19:49:14.947885 (+124114us) tablet_replica.cc:261] Starting instrumentation
0807 19:49:14.983777 (+ 35892us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":39620,"mutex_wait_us":3706,"raft.queue_time_us":2719,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6347,"tablet-copy.queue_time_us":463958,"tablet-download-pool.queue_time_us":6563,"tablet-download-pool.run_cpu_time_us":1645,"tablet-download-pool.run_wall_time_us":85471,"thread_start_us":557914,"threads_started":2}
I20260807 19:49:15.449986 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:15.450644 9893 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:15.462496 8902 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:15.470513 9008 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:15.497846 8775 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260807 19:49:15.487782 8737 consensus_peers.cc:597] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ba47e79caa54c409c53304a66ce0160. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:49:15.499162 8849 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.502269 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.503654 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.504175 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.502597 8778 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:15.504274 8779 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:15.505237 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.504053 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:15.500514 8849 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.511415 9023 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:15.517576 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.518142 8783 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:15.522166 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.500653 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:15.523638 10532 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.189s user 0.001s sys 0.000s
I20260807 19:49:15.503549 10372 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.524555 8849 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.523914 10359 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.528192 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.529028 9027 raft_consensus.cc:2393] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:15.530637 10516 log.cc:927] Time spent T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.296s user 0.000s sys 0.004s
I20260807 19:49:15.528046 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:15.531483 8901 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.531988 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.532208 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.532974 9021 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:15.533443 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.542670 8901 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:15.544018 9893 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:15.544812 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.545933 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:15.555116 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.558851 8849 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.536258 8908 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:15.567010 9019 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.542479 10372 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.573431 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:15.555338 10359 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:15.581197 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.585021 8853 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:15.580233 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 249892us
I20260807 19:49:15.595031 9838 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:15.596668 9838 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.597016 10539 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.598023 10539 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.598572 10539 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:15.599802 9838 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.600538 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.580601 10345 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.603076 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.602377 10372 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:15.582724 10359 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.621225 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.599668 10539 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.618160 9018 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
W20260807 19:49:15.641782 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 49163.9us
I20260807 19:49:15.624140 9027 raft_consensus.cc:2393] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:15.626185 10372 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:15.623026 10550 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:15.659055 8849 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.623131 10537 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.618667 10549 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260807 19:49:15.680440 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:15.716424 8849 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.714702 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.725934 10345 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:15.730638 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56954.2us
I20260807 19:49:15.731669 8849 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.662781 10550 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:15.734061 10550 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.705098 10430 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:15.721191 10536 log.cc:927] Time spent T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.102s user 0.000s sys 0.000s
I20260807 19:49:15.717762 10358 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:15.748142 10358 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.753158 10358 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.754629 9021 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.756888 8849 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.743417 10537 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.763761 9020 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:49:15.764765 10544 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.113s user 0.000s sys 0.001s
I20260807 19:49:15.756807 10372 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.765344 8849 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.778020 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.760452 10345 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.781296 9890 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:15.776971 10430 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:15.801190 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 452314us
I20260807 19:49:15.804123 10550 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:15.820549 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:15.828449 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 168390us
I20260807 19:49:15.809128 10336 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260807 19:49:15.816531 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 216815us
I20260807 19:49:15.838317 9017 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260807 19:49:15.843359 10517 log.cc:927] Time spent T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.517s user 0.000s sys 0.009s
I20260807 19:49:15.846475 8901 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:15.847289 10392 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:15.847477 10551 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:15.843170 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.849836 10522 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 19:49:15.855146 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 88664.1us
I20260807 19:49:15.854918 10522 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:15.859120 10522 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.861066 10321 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:15.862396 9044 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-300f918f185249e6895ab57edd060d31 received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:15.862939 9044 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-300f918f185249e6895ab57edd060d31 on tablet 300f918f185249e6895ab57edd060d31 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:15.863807 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:15.876545 8783 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:15.879806 8778 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.879993 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.880941 8788 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:15.884418 10557 consensus_queue.cc:1059] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:15.885608 10557 raft_consensus.cc:3055] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:15.886062 10557 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:15.886993 10557 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:15.887483 10550 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:15.890347 10359 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:15.893064 10557 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:15.902817 10364 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:15.903276 10364 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:15.904816 10502 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:15.906153 9023 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:15.906458 8968 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:15.878623 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:15.907732 8783 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:15.913240 8968 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:15.916985 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 73910.5us
I20260807 19:49:15.917783 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.926800 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:15.917943 8630 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:15.930850 8849 leader_election.cc:400] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 2. Current term is 3.
I20260807 19:49:15.931280 8849 leader_election.cc:403] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:15.934345 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.915514 10411 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:15.937928 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.926302 10451 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:15.946892 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.933650 10372 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 2. Current term is 3.
I20260807 19:49:15.945034 8628 catalog_manager.cc:5697] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:15.915994 10426 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260807 19:49:15.933733 10545 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.965270 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.931413 10408 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.960753 8628 catalog_manager.cc:5697] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:15.952759 8968 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:15.940191 10411 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:15.904240 10364 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:15.954380 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.953895 10525 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.949802 10451 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:15.954222 10554 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:15.960076 10430 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.945516 10359 consensus_queue.cc:1059] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260807 19:49:15.974751 10372 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.983603 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.948117 10392 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260807 19:49:15.988382 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:15.989101 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:15.971490 10411 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:15.978428 10358 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:15.952111 10513 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260807 19:49:15.994338 10364 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:15.995061 10556 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:15.982055 10451 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:15.996057 10359 raft_consensus.cc:3055] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:15.996903 10525 raft_consensus.cc:697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:15.952379 10522 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.002159 9023 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:15.986053 8628 catalog_manager.cc:5697] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:15.974406 10408 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.029989 8856 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:16.004788 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:15.989987 10550 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.009997 10513 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:16.012701 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.012023 10359 raft_consensus.cc:740] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:16.017939 8776 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:16.006681 10554 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.031847 10565 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.005441 10358 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:16.057483 10358 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:16.057053 10554 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.005957 10339 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.031193 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:16.048122 10523 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
W20260807 19:49:16.059424 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46773.7us
I20260807 19:49:16.064759 8789 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.069345 10580 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260807 19:49:16.072604 8740 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:16.073486 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:16.043098 10372 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:16.042729 10513 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:16.080888 10535 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.083280 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.092574 10345 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.060842 10550 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.060424 10430 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.097115 10430 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.097723 10430 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.095592 10550 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.104022 10592 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.104802 10592 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.107287 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.077520 10379 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.116513 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.076848 10372 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:16.083576 10556 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.077747 10339 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.127209 10525 consensus_queue.cc:237] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.128418 10359 consensus_queue.cc:260] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.130177 10565 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:16.145871 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 72451.9us
I20260807 19:49:16.146872 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:16.147964 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:16.149843 10379 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.150425 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:16.150517 10594 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.151234 10594 raft_consensus.cc:697] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.152606 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.154193 10538 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.148762 10584 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:16.152072 10379 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.155193 10584 raft_consensus.cc:697] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.156224 10584 consensus_queue.cc:237] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.152770 10557 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.154604 10585 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:16.160126 9020 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
W20260807 19:49:16.114197 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 43561.7us
I20260807 19:49:16.162817 9020 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.163041 9023 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.126000 10372 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:16.166504 9023 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.169546 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:16.131398 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 71756us
I20260807 19:49:16.163357 10430 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.185849 9893 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:16.152201 10594 consensus_queue.cc:237] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.190447 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.164147 10523 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:16.192409 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:16.207149 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.180462 10359 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:16.209182 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.205768 10565 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.193111 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.159791 10408 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.220525 10589 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.204988 10535 raft_consensus.cc:697] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.231473 10384 ts_tablet_manager.cc:933] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
I20260807 19:49:16.216729 10522 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.222846 10585 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.212970 10339 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.241953 10585 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.242659 10585 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:16.221196 10589 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:16.212865 10565 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.240306 10522 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.245136 8788 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:16.249886 10565 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.250448 10565 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:16.253115 10589 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:16.250967 10565 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.244609 10523 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.266409 10345 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:16.281265 10345 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.271838 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:16.269666 10336 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260807 19:49:16.295805 10345 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.300485 8788 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.232127 10535 consensus_queue.cc:237] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.302794 10452 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.310536 10372 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.314817 10411 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.317608 10452 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:16.325068 10452 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:16.327252 10592 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.371182 10451 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.354313 10556 raft_consensus.cc:697] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:16.380688 10556 consensus_queue.cc:237] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:16.384920 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 268530us
I20260807 19:49:16.385972 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.375953 10565 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.363685 10358 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.412231 10565 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.413070 10565 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.416368 10565 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.426738 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:16.418402 10359 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.430102 10565 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.429235 10513 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:16.548707 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 122065us
I20260807 19:49:16.472371 10557 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:16.560073 10513 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:16.585721 10379 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:16.586570 10379 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.588636 10372 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.589120 10358 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:16.589896 10358 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.590387 10358 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.543445 10411 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:16.483083 10565 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.615521 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:16.616021 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:16.587077 10512 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.634199 8910 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.638561 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.643604 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.641314 8911 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.647387 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.613684 10508 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.651151 8915 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:16.653009 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.655413 8902 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260807 19:49:16.659861 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:16.640225 10550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s user 0.017s sys 0.022s
I20260807 19:49:16.661993 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:16.661909 10379 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.664274 10508 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.675629 9019 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:49:16.666087 10411 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:16.666848 10565 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.667548 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:16.689966 8913 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:16.668818 10364 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260807 19:49:16.670109 10358 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:16.670859 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.654649 10580 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260807 19:49:16.637168 10372 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.619786 10512 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.687923 9023 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:49:16.674358 10550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s user 0.018s sys 0.023s
I20260807 19:49:16.699777 10591 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.698256 10451 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:16.701071 10508 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:16.727259 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:16.736243 10595 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.736939 10595 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.737509 9019 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:16.732085 9022 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260807 19:49:16.720203 10359 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.705987 10372 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.745281 8856 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:16.749373 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:16.750270 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:16.751052 8915 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:16.755678 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:16.755968 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:16.756656 8901 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:16.758448 8901 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:16.729874 10557 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.728775 10430 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.785733 10430 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.765219 10345 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:16.779079 10384 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 0.547s user 0.010s sys 0.009s
I20260807 19:49:16.780882 10537 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:16.801885 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:16.775134 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:16.721072 10591 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:16.788717 10425 log.cc:927] Time spent T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Append to log took a long time: real 0.119s user 0.000s sys 0.001s
I20260807 19:49:16.795696 10512 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:16.796495 10565 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:16.797775 9893 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260807 19:49:16.800242 8902 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:16.817998 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:16.847554 8779 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:16.823202 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:16.848866 8901 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:16.822889 8906 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:16.823964 8913 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:49:16.803635 10384 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 0.571s user 0.010s sys 0.009s
I20260807 19:49:16.791356 10372 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.862084 10372 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:16.862267 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:16.862896 8906 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:16.862731 10372 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.804637 10345 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.848811 8903 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:16.865228 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:16.835078 8740 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s user 0.029s sys 0.046s
I20260807 19:49:16.865885 8910 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:16.821027 10594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s user 0.024s sys 0.043s
W20260807 19:49:16.871402 8740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s user 0.029s sys 0.046s
I20260807 19:49:16.829617 10430 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.876307 8779 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:16.771883 10359 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.827687 10537 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:16.798766 10557 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.804505 10364 consensus_queue.cc:1059] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
W20260807 19:49:16.944491 10598 log.cc:927] Time spent T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.228s user 0.000s sys 0.001s
I20260807 19:49:16.954581 10591 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.956220 10359 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:16.856848 10565 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:16.937260 10384 tablet_copy_client.cc:323] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.193:33185
I20260807 19:49:16.977025 8906 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:16.989741 8910 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:16.876403 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 667224us
W20260807 19:49:16.874846 10594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s user 0.024s sys 0.044s
I20260807 19:49:16.816215 10595 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.050494 8915 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260807 19:49:17.084609 8778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s user 0.021s sys 0.045s
W20260807 19:49:17.085067 8778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s user 0.021s sys 0.045s
W20260807 19:49:17.060143 10339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s user 0.013s sys 0.049s
W20260807 19:49:17.091053 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 85308.1us
W20260807 19:49:17.090852 10339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s user 0.014s sys 0.049s
I20260807 19:49:17.090986 10539 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260807 19:49:17.090495 10372 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.090497 10565 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.089532 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:17.095162 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:17.146266 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:17.099771 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:17.103628 8970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s user 0.003s sys 0.037s
W20260807 19:49:17.151615 8970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s user 0.003s sys 0.037s
I20260807 19:49:17.155418 10580 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.156128 10580 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:17.162176 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:17.097388 10615 log.cc:927] Time spent T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.134s user 0.000s sys 0.001s
I20260807 19:49:17.084738 10530 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:17.186620 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 441388us
W20260807 19:49:17.062757 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 195906us
W20260807 19:49:17.183480 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 88392.5us
W20260807 19:49:17.152163 8776 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s user 0.014s sys 0.051s
I20260807 19:49:17.097018 10595 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:17.145872 9888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.842s user 0.004s sys 0.083s
I20260807 19:49:17.164508 10508 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:16.963104 10537 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.225059 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.225804 9017 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:17.193836 10321 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:17.228638 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 62108.5us
I20260807 19:49:17.231107 9017 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260807 19:49:17.231545 9017 pending_rounds.cc:85] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Aborting all ops after (but not including) 4
I20260807 19:49:17.220180 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:17.184863 10339 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.232869 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:17.234197 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:17.202529 9888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s user 0.004s sys 0.084s
I20260807 19:49:17.231765 9017 pending_rounds.cc:107] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260807 19:49:17.251967 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:17.252503 9017 raft_consensus.cc:2889] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:17.250409 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.254762 9023 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
W20260807 19:49:17.277029 8780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s user 0.017s sys 0.021s
W20260807 19:49:17.277887 8780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s user 0.018s sys 0.022s
W20260807 19:49:17.219592 10408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.846s user 0.012s sys 0.053s
I20260807 19:49:17.240679 10523 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 19:49:17.272166 8904 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s user 0.005s sys 0.021s
I20260807 19:49:17.193267 10530 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.253619 10594 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.240324 10321 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:17.298270 8783 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s user 0.008s sys 0.022s
W20260807 19:49:17.298799 8783 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.637s user 0.008s sys 0.022s
I20260807 19:49:17.241382 10508 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.240370 10595 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:17.286846 10408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s user 0.013s sys 0.054s
W20260807 19:49:17.287158 10556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s user 0.008s sys 0.019s
W20260807 19:49:17.287362 10603 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.599s user 0.009s sys 0.077s
W20260807 19:49:17.365777 10603 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.681s user 0.009s sys 0.078s
I20260807 19:49:17.367357 10595 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:17.301801 8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.070s sys 0.059s
I20260807 19:49:17.291252 8856 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:17.245783 10372 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260807 19:49:17.280463 10578 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s user 0.018s sys 0.014s
W20260807 19:49:17.325958 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 50774.4us
I20260807 19:49:17.287932 10565 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:17.283751 8904 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s user 0.005s sys 0.021s
W20260807 19:49:17.361335 10556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s user 0.008s sys 0.019s
I20260807 19:49:17.287410 10339 consensus_queue.cc:1059] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260807 19:49:17.302130 10557 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:17.369086 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 77940.4us
I20260807 19:49:17.374704 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:17.318190 10591 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:17.377981 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.380970 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.395916 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:17.380663 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:17.368609 8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s user 0.078s sys 0.067s
I20260807 19:49:17.320066 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 43) took 1112 ms. Trace:
W20260807 19:49:17.543061 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 930432us
I20260807 19:49:17.545079 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.375808 8908 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:17.318758 10554 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:17.553313 8856 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:17.552134 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:17.527086 10604 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.291s user 0.000s sys 0.001s
W20260807 19:49:17.470598 10606 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.128s user 0.001s sys 0.000s
I20260807 19:49:17.379729 10364 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:17.369025 10612 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.146s user 0.000s sys 0.001s
I20260807 19:49:17.370942 10372 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:17.372982 10578 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s user 0.018s sys 0.015s
W20260807 19:49:17.382519 10618 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.184s user 0.000s sys 0.001s
W20260807 19:49:17.581609 10613 log.cc:927] Time spent T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.175s user 0.000s sys 0.000s
I20260807 19:49:17.552100 10591 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:17.378928 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.583109 9008 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:17.584438 10591 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.585278 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 41) took 1395 ms. Trace:
I20260807 19:49:17.586792 10651 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:17.585702 8780 rpcz_store.cc:276] 0807 19:49:16.189479 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.189613 (+ 134us) service_pool.cc:224] Handling call
0807 19:49:16.189968 (+ 355us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.194674 (+ 4706us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.194709 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.209778 (+ 15069us) log.cc:844] Serialized 26 byte log entry
0807 19:49:16.692234 (+482456us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:17.278269 (+586035us) log.cc:844] Serialized 14 byte log entry
0807 19:49:17.320210 (+ 41941us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.320405 (+ 195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:17.585032 (+264627us) raft_consensus.cc:1685] finished
0807 19:49:17.585056 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.585211 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41055,"prepare.queue_time_us":668414,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":4211,"raft.queue_time_us":50821,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":38797,"thread_start_us":1037813,"threads_started":2,"wal-append.queue_time_us":847296}
I20260807 19:49:17.587224 10651 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:17.381435 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.538750 10634 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:17.588380 10634 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.374150 10580 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:17.405854 10359 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.379513 10321 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.368274 10595 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:17.381745 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:17.571480 10364 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.579465 10372 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:17.563727 10554 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:17.587687 10651 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:17.376187 9888 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33436 (request call id 105) took 1098 ms. Trace:
I20260807 19:49:17.552023 10565 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.615267 10372 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:17.614820 10634 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:17.631716 10345 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:17.636489 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:17.636682 10530 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:17.606330 10591 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260807 19:49:17.618187 9888 rpcz_store.cc:276] 0807 19:49:16.277403 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.278284 (+ 881us) service_pool.cc:224] Handling call
0807 19:49:16.300939 (+ 22655us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.301508 (+ 569us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.301546 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.302192 (+ 646us) log.cc:844] Serialized 26 byte log entry
0807 19:49:17.262664 (+960472us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:17.273467 (+ 10803us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.282298 (+ 8831us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:17.375907 (+ 93609us) raft_consensus.cc:1685] finished
0807 19:49:17.375926 (+ 19us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.376102 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3101,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":2957,"thread_start_us":939344,"threads_started":1,"wal-append.queue_time_us":969147}
I20260807 19:49:17.640784 10554 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:17.605937 10359 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.608856 10595 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:17.631803 10614 log.cc:927] Time spent T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260807 19:49:17.648437 10345 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:17.671813 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 128007us
I20260807 19:49:17.663193 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:17.701264 8972 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.704084 8972 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.707814 8972 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.665869 10508 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:17.701964 10557 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.709280 8972 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.711719 8972 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:17.712601 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 152251us
W20260807 19:49:17.712643 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 170261us
I20260807 19:49:17.716760 8972 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.681123 10345 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:17.772832 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 56169.9us
I20260807 19:49:17.745908 8856 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:17.773699 8972 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.709084 10580 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.711023 10392 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:17.707217 10530 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:17.796582 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 161909us
I20260807 19:49:17.861065 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 42) took 1671 ms. Trace:
I20260807 19:49:17.705049 10638 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260807 19:49:17.863677 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 87702.1us
I20260807 19:49:17.812885 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 58) took 1128 ms. Trace:
I20260807 19:49:17.865063 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 999) took 1143 ms. Trace:
W20260807 19:49:17.865890 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 491261us
I20260807 19:49:17.883574 8783 rpcz_store.cc:276] 0807 19:49:16.189807 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.189917 (+ 110us) service_pool.cc:224] Handling call
0807 19:49:16.190190 (+ 273us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.190478 (+ 288us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.190505 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.196533 (+ 6028us) log.cc:844] Serialized 26 byte log entry
0807 19:49:16.640877 (+444344us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:17.299249 (+658372us) log.cc:844] Serialized 14 byte log entry
0807 19:49:17.860300 (+561051us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b1800054b80. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f6878477d1a 00007f6881c0b5d9 00007f68785003a0 00007f68784fae94 00007f68784fae05 00007f687b5fc440 00007f687b60eaf6 00007f687b61f667 00007f687b61f5e8
0807 19:49:17.860595 (+ 295us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.860687 (+ 92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:17.860726 (+ 39us) raft_consensus.cc:1685] finished
0807 19:49:17.860835 (+ 109us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.861001 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37410,"prepare.queue_time_us":692680,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":581832,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":3881,"spinlock_wait_cycles":10702720,"thread_start_us":1052054,"threads_started":2,"wal-append.queue_time_us":645921}
I20260807 19:49:17.867125 10334 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.867059 10339 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.867394 10556 raft_consensus.cc:3055] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
W20260807 19:49:17.868183 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 220475us
I20260807 19:49:17.872390 10379 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.548666 8776 rpcz_store.cc:276] 0807 19:49:16.190085 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.191814 (+ 1729us) service_pool.cc:224] Handling call
0807 19:49:16.213653 (+ 21839us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.220571 (+ 6918us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.220602 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.221156 (+ 554us) log.cc:844] Serialized 26 byte log entry
0807 19:49:16.560577 (+339421us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:17.240610 (+680033us) log.cc:844] Serialized 14 byte log entry
0807 19:49:17.261294 (+ 20684us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.279832 (+ 18538us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:17.284146 (+ 4314us) raft_consensus.cc:1685] finished
0807 19:49:17.287523 (+ 3377us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.298572 (+ 11049us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13228,"prepare.queue_time_us":721727,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":6188,"raft.queue_time_us":10349,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":13517,"thread_start_us":843859,"threads_started":2,"wal-append.queue_time_us":499174}
I20260807 19:49:17.846799 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1008) took 1073 ms. Trace:
I20260807 19:49:17.823424 10537 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.742482 10508 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:17.711849 10557 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.808321 10647 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:17.827024 10667 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.773981 10364 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:17.710901 10578 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 19:49:17.828444 10625 log.cc:927] Time spent T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.116s user 0.000s sys 0.001s
W20260807 19:49:17.829326 10639 log.cc:927] Time spent T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.126s user 0.000s sys 0.001s
I20260807 19:49:17.889832 10677 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.883235 10321 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:17.894686 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:17.859257 10359 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.870159 10554 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:17.902870 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 129241us
I20260807 19:49:17.898849 10535 consensus_queue.cc:1059] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260807 19:49:17.906090 10530 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:17.902448 8789 rpcz_store.cc:276] 0807 19:49:16.773515 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.795616 (+ 22101us) service_pool.cc:224] Handling call
0807 19:49:16.795924 (+ 308us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.796265 (+ 341us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.796293 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.798195 (+ 1902us) log.cc:844] Serialized 26 byte log entry
0807 19:49:17.280302 (+482107us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:17.281296 (+ 994us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.281377 (+ 81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:17.846501 (+565124us) raft_consensus.cc:1685] finished
0807 19:49:17.846527 (+ 26us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.846712 (+ 185us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1142,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":481595,"threads_started":1,"wal-append.queue_time_us":913765}
I20260807 19:49:17.905436 10451 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.909746 10674 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.913157 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
W20260807 19:49:17.940428 10629 log.cc:927] Time spent T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.579s user 0.001s sys 0.004s
I20260807 19:49:17.911377 10667 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.921504 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:17.905531 10647 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:17.933434 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.887125 10595 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.952049 10595 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.952601 10595 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.911789 10345 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.961845 10345 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:17.962469 10345 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.897972 10556 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:17.965305 10556 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.966495 10364 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:17.966964 10556 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:17.967396 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.951361 10647 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:17.968253 10364 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:17.968868 10364 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:17.970170 10364 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.905921 10537 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.961030 10595 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.930049 10508 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:17.932927 10523 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:17.979158 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.981103 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:17.941294 10674 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.987012 8972 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:17.952651 10638 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:17.998464 10408 consensus_queue.cc:1059] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:49:17.998693 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 47289.7us
I20260807 19:49:17.999580 10557 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.000475 10667 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.001803 10674 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:17.988893 10607 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:18.003497 10358 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:17.933197 10321 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:17.940265 10554 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:17.982056 10634 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:18.017820 10600 consensus_queue.cc:1059] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260807 19:49:17.992750 10451 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:17.992547 10392 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:17.994740 10638 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:17.997437 10660 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:17.933358 10677 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:18.030444 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 76840us
I20260807 19:49:18.015419 10512 raft_consensus.cc:3055] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:17.988967 10339 consensus_queue.cc:1059] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260807 19:49:18.003504 10372 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:17.988775 10662 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260807 19:49:18.046195 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.047004 8915 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:18.055090 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.058980 8901 raft_consensus.cc:1275] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:18.028426 10638 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:18.016521 10535 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260807 19:49:17.887185 8904 rpcz_store.cc:276] 0807 19:49:16.679930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.683728 (+ 3798us) service_pool.cc:224] Handling call
0807 19:49:16.684189 (+ 461us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:16.684524 (+ 335us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 19:49:17.562836 (+878312us) log.cc:844] Serialized 14 byte log entry
0807 19:49:17.741537 (+178701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:17.770151 (+ 28614us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:17.774876 (+ 4725us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.797935 (+ 23059us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.806248 (+ 8313us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":975128,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":22726,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":34832,"thread_start_us":971124,"threads_started":2,"wal-append.queue_time_us":252118}
I20260807 19:49:18.064767 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:18.070385 8902 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:18.037598 10345 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.065131 8906 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:18.039718 10512 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:18.030601 10451 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:18.030229 10530 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.064327 8908 raft_consensus.cc:1275] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:18.044473 10408 raft_consensus.cc:3055] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:18.015622 10525 consensus_queue.cc:1059] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260807 19:49:18.072955 8901 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:17.892920 8775 rpcz_store.cc:276] 0807 19:49:16.721287 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.723008 (+ 1721us) service_pool.cc:224] Handling call
0807 19:49:16.741353 (+ 18345us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:16.741593 (+ 240us) raft_consensus.cc:1534] Early marking committed up to index 2
0807 19:49:17.246338 (+504745us) log.cc:844] Serialized 14 byte log entry
0807 19:49:17.864636 (+618298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:17.864703 (+ 67us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:17.864764 (+ 61us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.864857 (+ 93us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:17.864992 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366884,"prepare.queue_time_us":628148,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":369158,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":712415,"threads_started":2,"wal-append.queue_time_us":520560}
I20260807 19:49:18.037227 10523 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260807 19:49:18.059134 10651 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.204037 8628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38436 (request call id 194) took 2916 ms. Trace:
I20260807 19:49:18.117480 10688 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260807 19:49:18.207589 10681 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.127781 10554 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:18.208261 10681 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:18.141981 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 111736us
I20260807 19:49:18.069258 10508 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.204749 8628 rpcz_store.cc:276] 0807 19:49:15.287336 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:15.287454 (+ 118us) service_pool.cc:224] Handling call
0807 19:49:18.203963 (+2916509us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:16.187389 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:16.187909 (+ 520us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:16.187935 (+ 26us) write_op.cc:435] Acquired schema lock
0807 19:49:16.187947 (+ 12us) tablet.cc:662] Decoding operations
0807 19:49:16.188517 (+ 570us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:16.194044 (+ 5527us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:16.205094 (+ 11050us) tablet.cc:685] Acquiring locks for 3 operations
0807 19:49:16.227411 (+ 22317us) tablet.cc:701] Row locks acquired
0807 19:49:16.228707 (+ 1296us) write_op.cc:260] PREPARE: finished
0807 19:49:16.236814 (+ 8107us) write_op.cc:270] Start()
0807 19:49:16.240093 (+ 3279us) write_op.cc:276] Timestamp: P: 1786132156235180 usec, L: 0
0807 19:49:16.240815 (+ 722us) op_driver.cc:348] REPLICATION: starting
0807 19:49:16.267539 (+ 26724us) log.cc:844] Serialized 1307 byte log entry
0807 19:49:17.367024 (+1099485us) op_driver.cc:464] REPLICATION: finished
0807 19:49:17.583381 (+216357us) write_op.cc:301] APPLY: starting
0807 19:49:17.583515 (+ 134us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:17.583750 (+ 235us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:17.583941 (+ 191us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:17.603143 (+ 19202us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:17.622124 (+ 18981us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0807 19:49:17.628126 (+ 6002us) write_op.cc:312] APPLY: finished
0807 19:49:17.637267 (+ 9141us) log.cc:844] Serialized 34 byte log entry
0807 19:49:17.637467 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:17.637573 (+ 106us) write_op.cc:454] Released schema lock
0807 19:49:17.637624 (+ 51us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":544009,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":216086,"apply.run_cpu_time_us":2706,"apply.run_wall_time_us":54589,"mutex_wait_us":3867,"num_ops":3,"prepare.queue_time_us":125084,"prepare.run_cpu_time_us":4042,"prepare.run_wall_time_us":80465,"replication_time_us":1125978,"thread_start_us":288815,"threads_started":1}]]}
I20260807 19:49:18.178229 10591 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:18.171867 10656 log.cc:927] Time spent T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.455s user 0.000s sys 0.002s
I20260807 19:49:18.129374 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:18.182212 8629 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 2 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:18.162757 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 965) took 1361 ms. Trace:
I20260807 19:49:18.080952 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.306326 8915 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:18.155102 10647 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.316383 10708 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:18.213841 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 112372us
W20260807 19:49:18.231092 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 70374.7us
I20260807 19:49:18.083784 10660 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:18.318459 10691 raft_consensus.cc:3055] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
W20260807 19:49:18.112581 10650 log.cc:927] Time spent T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.528s user 0.005s sys 0.000s
W20260807 19:49:18.166649 10632 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.847s user 0.004s sys 0.000s
I20260807 19:49:18.110441 10321 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:18.185274 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38426 (request call id 205) took 2417 ms. Trace:
I20260807 19:49:18.300048 10591 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.331903 8630 rpcz_store.cc:276] 0807 19:49:15.767986 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:15.786762 (+ 18776us) service_pool.cc:224] Handling call
0807 19:49:18.185126 (+2398364us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:16.090035 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:16.129776 (+ 39741us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:16.129813 (+ 37us) write_op.cc:435] Acquired schema lock
0807 19:49:16.129825 (+ 12us) tablet.cc:662] Decoding operations
0807 19:49:16.130163 (+ 338us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:16.130215 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:16.130242 (+ 27us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:49:16.130437 (+ 195us) tablet.cc:701] Row locks acquired
0807 19:49:16.130456 (+ 19us) write_op.cc:260] PREPARE: finished
0807 19:49:16.130597 (+ 141us) write_op.cc:270] Start()
0807 19:49:16.135553 (+ 4956us) write_op.cc:276] Timestamp: P: 1786132156130552 usec, L: 0
0807 19:49:16.160993 (+ 25440us) op_driver.cc:348] REPLICATION: starting
0807 19:49:16.186924 (+ 25931us) log.cc:844] Serialized 532 byte log entry
0807 19:49:16.684377 (+497453us) op_driver.cc:464] REPLICATION: finished
0807 19:49:17.561221 (+876844us) write_op.cc:301] APPLY: starting
0807 19:49:17.567108 (+ 5887us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:17.580853 (+ 13745us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:17.581084 (+ 231us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:17.581403 (+ 319us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:17.581620 (+ 217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:49:17.581716 (+ 96us) write_op.cc:312] APPLY: finished
0807 19:49:17.582022 (+ 306us) log.cc:844] Serialized 22 byte log entry
0807 19:49:17.582270 (+ 248us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:17.582354 (+ 84us) write_op.cc:454] Released schema lock
0807 19:49:17.582400 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:17.582889 (+ 489us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:17.582942 (+ 53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":695,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":844567,"apply.run_cpu_time_us":2492,"apply.run_wall_time_us":31258,"num_ops":1,"prepare.queue_time_us":134730,"prepare.run_cpu_time_us":3123,"prepare.run_wall_time_us":115588,"replication_time_us":523124,"thread_start_us":740863,"threads_started":2}]]}
I20260807 19:49:18.105888 8902 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:18.111919 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.229857 10359 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:18.126534 10678 log.cc:927] Time spent T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.100s user 0.001s sys 0.000s
I20260807 19:49:18.317534 8911 rpcz_store.cc:276] 0807 19:49:16.801446 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.801591 (+ 145us) service_pool.cc:224] Handling call
0807 19:49:18.162462 (+1360871us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002fba54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:18.162644 (+ 182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2347490176}
I20260807 19:49:18.151436 10607 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:18.336834 10607 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.336551 10359 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:18.298100 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 228630us
I20260807 19:49:18.338042 8908 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:18.148763 10601 raft_consensus.cc:3055] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:18.188083 10600 raft_consensus.cc:3055] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:18.119220 10537 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.353534 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:18.131939 10372 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.141041 10345 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.111362 10565 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.230201 10408 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.374050 10683 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.374260 10359 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:18.374903 10408 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.347996 10677 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.360139 10601 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.375993 10408 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:18.328281 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 203688us
I20260807 19:49:18.354315 8901 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:18.348192 10674 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.354383 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.251870 10595 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.123997 10626 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.347s
I20260807 19:49:18.350613 10691 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.362963 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:18.416071 8789 raft_consensus.cc:2393] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:18.363897 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:18.346558 10554 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:18.330127 10694 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.372493 10523 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.331000 10681 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.350840 10594 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.351074 10600 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.373318 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.373788 8775 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:18.167871 10638 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.377691 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1002) took 1644 ms. Trace:
I20260807 19:49:18.352707 10705 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.353363 8915 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:18.429266 10578 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.448302 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.447415 8787 rpcz_store.cc:276] 0807 19:49:16.732839 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.732993 (+ 154us) service_pool.cc:224] Handling call
0807 19:49:16.789373 (+ 56380us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:16.789600 (+ 227us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.802933 (+ 13333us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.802966 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.803450 (+ 484us) log.cc:844] Serialized 26 byte log entry
0807 19:49:17.289368 (+485918us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:17.289416 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.289493 (+ 77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:18.111831 (+822338us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:18.377462 (+265631us) raft_consensus.cc:1685] finished
0807 19:49:18.377486 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:18.377630 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39760,"raft.queue_time_us":407,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":32684,"spinlock_wait_cycles":41639552,"thread_start_us":468586,"threads_started":1,"wal-append.queue_time_us":763356}
I20260807 19:49:18.449129 8915 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:18.379231 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.376784 10601 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.426932 10554 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.380909 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:18.383244 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.356293 8903 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:18.388181 8911 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:18.391152 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:18.406175 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.350548 10379 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.409605 10703 raft_consensus.cc:3055] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:18.382500 10662 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.410029 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:18.410456 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:18.410748 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:18.411163 10591 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.360580 10607 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.411082 10595 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.417457 8901 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:18.430392 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 995) took 2209 ms. Trace:
I20260807 19:49:18.361464 10339 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.130438 10512 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.379896 10556 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.436066 10686 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:18.411208 10580 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:18.436849 10692 raft_consensus.cc:1064] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:18.129832 10451 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.150492 10392 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.429419 10508 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:18.432997 10411 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:18.410456 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.439121 10523 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.377576 10345 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.372843 10702 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.440788 8908 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:18.450457 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:18.447165 10709 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
I20260807 19:49:18.380020 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:18.452565 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:18.456254 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 65200us
I20260807 19:49:18.442695 10712 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:18.453481 8849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.535s (SENT) [suppressed 43 similar messages]
I20260807 19:49:18.440040 10647 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.461515 10379 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.507257 10379 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.440066 10651 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.329133 10708 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.428087 10674 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.443063 10675 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.464428 9850 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:18.437302 10683 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.447065 10705 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.460897 10364 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:18.443440 10594 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.481336 8778 rpcz_store.cc:276] 0807 19:49:16.208405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.208610 (+ 205us) service_pool.cc:224] Handling call
0807 19:49:16.428295 (+219685us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:16.428603 (+ 308us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:16.429024 (+ 421us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:16.429056 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:16.429630 (+ 574us) log.cc:844] Serialized 26 byte log entry
0807 19:49:17.085425 (+655795us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:17.085469 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:17.085538 (+ 69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:18.165805 (+1080267us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:18.410840 (+245035us) raft_consensus.cc:1685] finished
0807 19:49:18.410863 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:18.417388 (+ 6525us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18909,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":23340,"spinlock_wait_cycles":89723136,"thread_start_us":655352,"threads_started":1,"wal-append.queue_time_us":816870}
I20260807 19:49:18.514766 8904 raft_consensus.cc:2393] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:18.471987 10601 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:18.472203 10703 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:18.491959 10334 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:18.448741 10691 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:18.499783 8902 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:18.503186 8972 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:18.494012 10607 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:18.487747 10339 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.492578 10556 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.454368 10660 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.483935 10662 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.496951 10411 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.448736 10578 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.493324 10580 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.446000 10694 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.495301 10600 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.536885 10695 raft_consensus.cc:1064] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
W20260807 19:49:18.533684 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 69398us
I20260807 19:49:18.539160 9021 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:18.496116 10686 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.492136 10554 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.511958 10688 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
W20260807 19:49:18.547425 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 44308.1us
W20260807 19:49:18.572932 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.497056 10508 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.578467 10345 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.579145 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:18.537259 10647 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:18.577219 10634 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:18.566612 10667 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.566186 10638 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:18.547866 10334 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.597788 9021 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:18.578204 10692 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.519439 10712 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.587503 10600 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:18.503750 10702 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.601408 10411 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:18.633459 8968 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:18.591341 10647 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.619509 10554 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:18.617658 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
W20260807 19:49:18.649852 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.594645 10667 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.534171 10675 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.681020 10683 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.629506 10708 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.684736 10578 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:18.667047 10411 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:18.696257 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:18.710992 8787 raft_consensus.cc:2393] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:18.676846 10523 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.726303 10674 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.741150 10702 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:18.742307 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 288864us
I20260807 19:49:18.753466 10339 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.744194 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:18.652653 10703 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.782131 10703 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:18.753170 10594 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.739529 10556 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.787506 8972 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:18.791059 10411 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:18.791607 10411 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:18.798028 10411 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:18.798751 10694 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:18.775893 10705 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.789135 8788 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:18.809369 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:18.809762 10634 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:18.832916 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.812278 10638 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.810055 10729 consensus_queue.cc:1059] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.178s
I20260807 19:49:18.861157 10556 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:18.872865 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:18.888773 8740 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s user 0.025s sys 0.073s
W20260807 19:49:18.889159 8740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s user 0.025s sys 0.073s
I20260807 19:49:18.885447 10730 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.884240 10594 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:18.886181 10667 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:18.887001 10638 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.884552 10647 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:18.909797 10662 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:18.913884 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:18.911177 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 264135us
I20260807 19:49:18.926254 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:18.936535 9021 raft_consensus.cc:2393] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:18.924553 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.889587 10556 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:18.940637 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:18.911505 10730 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.945078 9021 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:18.920684 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:18.915630 10594 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:18.946624 8856 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:18.916832 10600 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:18.934281 10549 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:18.917553 10339 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260807 19:49:18.935829 10675 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:18.973387 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.974005 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:18.971647 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:18.980603 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:18.981279 9021 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:18.981701 10730 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:18.982669 8856 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:18.983834 10667 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:18.984381 10667 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.985055 10667 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:18.985584 10732 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.114s user 0.001s sys 0.000s
W20260807 19:49:18.986474 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:18.987543 9023 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s user 0.024s sys 0.043s
I20260807 19:49:18.963122 10647 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:18.990095 10647 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:18.967141 10549 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:18.969038 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:19.006664 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.007434 9021 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:19.008510 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.010202 8856 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:18.990836 9023 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s user 0.024s sys 0.043s
I20260807 19:49:18.992745 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:18.966272 10600 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:19.016494 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 49292.2us
I20260807 19:49:18.993898 10647 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.021871 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.021807 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:19.005858 10549 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:19.046219 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.047032 9021 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:19.048038 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.049911 9021 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:19.052071 8856 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:19.053098 8856 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:19.057415 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.020982 10702 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:19.020731 10705 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:19.064023 9023 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:18.974881 10728 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260807 19:49:19.072019 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.075073 9021 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:19.076553 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.076920 8856 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:19.080950 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:19.081688 9023 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:19.058203 10601 consensus_meta.cc:287] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.520s user 0.006s sys 0.007s
I20260807 19:49:19.079764 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.090975 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:19.092064 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:19.083372 8856 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:19.065800 10675 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:19.075934 10408 consensus_meta.cc:287] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.699s user 0.008s sys 0.007s
I20260807 19:49:19.099670 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.117018 8915 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:19.073374 10674 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:19.076412 10705 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:19.122726 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.064524 10379 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:19.124125 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.129280 9023 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:19.100447 10727 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:19.099256 10702 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:19.096072 10708 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:19.137238 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:19.100798 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 365624us
I20260807 19:49:19.142134 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.116542 10594 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:19.103667 10675 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:19.119879 10674 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:19.135744 10705 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:19.123445 10683 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:19.137677 10727 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:19.155515 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.166628 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:19.167459 9023 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.158334 8915 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:19.122296 10638 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:19.142313 10748 log.cc:927] Time spent T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.157s user 0.000s sys 0.002s
W20260807 19:49:19.177335 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:19.138077 10379 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.171792 10702 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:19.199445 10411 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.163666 10683 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:19.156637 10675 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.156204 10674 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.169729 10727 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:19.205497 10607 consensus_meta.cc:287] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.638s user 0.000s sys 0.013s
I20260807 19:49:19.148801 10556 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:19.221817 8901 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s user 0.035s sys 0.026s
W20260807 19:49:19.222929 8901 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.575s user 0.035s sys 0.026s
I20260807 19:49:19.223515 9023 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:19.232829 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:19.217952 10753 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.083s user 0.000s sys 0.001s
W20260807 19:49:19.217739 10750 log.cc:927] Time spent T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.142s user 0.001s sys 0.000s
I20260807 19:49:19.235848 9023 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:19.236544 9021 raft_consensus.cc:2393] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:19.228379 10751 log.cc:927] Time spent T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260807 19:49:19.224881 10683 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.244295 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:19.253429 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 170120us
I20260807 19:49:19.254336 8856 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:19.255551 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 163549us
I20260807 19:49:19.262982 9020 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:19.267892 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:19.268041 10408 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:19.269616 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:19.269817 10594 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:19.270334 10601 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:19.270678 10556 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:19.271224 10523 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:19.271829 10556 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:19.269642 10408 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 19:49:19.273319 10408 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:19.273725 10408 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:19.269660 9893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33434 (request call id 82) took 1225 ms. Trace:
I20260807 19:49:19.272260 10556 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:19.277356 10556 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.279464 9027 raft_consensus.cc:1275] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:19.275331 9893 rpcz_store.cc:276] 0807 19:49:18.043927 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.062931 (+ 19004us) service_pool.cc:224] Handling call
0807 19:49:18.101280 (+ 38349us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:18.134747 (+ 33467us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 19:49:18.160637 (+ 25890us) log.cc:844] Serialized 14 byte log entry
0807 19:49:19.269152 (+1108515us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18002c1580. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f687b1ab6ac 00007f68784e0e3b 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fae05 00007f687b5fc440 00007f687b60eaf6 00007f687b61f667
0807 19:49:19.269267 (+ 115us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:19.269324 (+ 57us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:19.269369 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:19.269450 (+ 81us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:19.269591 (+ 141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75696,"prepare.queue_time_us":10974,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":152727,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":15281,"spinlock_wait_cycles":26282368,"thread_start_us":35175,"threads_started":1}
I20260807 19:49:19.280905 9027 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:19.278644 10556 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.283880 9027 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:19.284791 10408 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.286635 10408 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.292042 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:19.298908 10594 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:19.299659 10594 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.302150 10594 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:19.303834 10595 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:19.312301 10595 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:19.312979 10595 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.285939 9018 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:19.317255 10712 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:19.318986 10392 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:19.319697 10392 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:19.320654 10392 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.284965 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.321792 9008 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:19.311510 10601 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:19.324852 10601 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:19.321012 10334 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:19.326298 10601 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.327797 9017 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:19.330775 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:19.331110 9017 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:19.332996 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:19.333989 9017 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:19.334314 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:19.336886 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:19.337143 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 45335.8us
I20260807 19:49:19.337670 10508 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:19.337931 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:19.343770 8972 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:19.344672 10662 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:19.345163 10662 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:19.345700 10662 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:19.349510 10358 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:19.350034 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:19.350885 10358 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:19.351934 8972 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:19.318631 10595 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:19.354585 10595 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:19.355170 10595 raft_consensus.cc:697] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:19.355962 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 65) took 3084 ms. Trace:
I20260807 19:49:19.358682 10580 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:19.364380 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:19.365804 8972 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:19.252627 10554 consensus_meta.cc:287] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.586s user 0.012s sys 0.000s
I20260807 19:49:19.349385 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:19.362699 10600 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:19.384438 10379 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:19.359755 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:19.356498 8629 rpcz_store.cc:276] 0807 19:49:16.271084 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.271264 (+ 180us) service_pool.cc:224] Handling call
0807 19:49:19.355883 (+3084619us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:18.333196 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:18.333902 (+ 706us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:18.333936 (+ 34us) write_op.cc:435] Acquired schema lock
0807 19:49:18.333954 (+ 18us) tablet.cc:662] Decoding operations
0807 19:49:18.334299 (+ 345us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:18.334383 (+ 84us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:18.334428 (+ 45us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:49:18.334677 (+ 249us) tablet.cc:701] Row locks acquired
0807 19:49:18.334698 (+ 21us) write_op.cc:260] PREPARE: finished
0807 19:49:18.334878 (+ 180us) write_op.cc:270] Start()
0807 19:49:18.335006 (+ 128us) write_op.cc:276] Timestamp: P: 1786132158334821 usec, L: 0
0807 19:49:18.335033 (+ 27us) op_driver.cc:348] REPLICATION: starting
0807 19:49:18.335779 (+ 746us) log.cc:844] Serialized 532 byte log entry
0807 19:49:18.343154 (+ 7375us) op_driver.cc:464] REPLICATION: finished
0807 19:49:18.625783 (+282629us) write_op.cc:301] APPLY: starting
0807 19:49:18.625940 (+ 157us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:18.626104 (+ 164us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:18.626305 (+ 201us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:18.626579 (+ 274us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:18.626777 (+ 198us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:49:18.626837 (+ 60us) write_op.cc:312] APPLY: finished
0807 19:49:18.627227 (+ 390us) log.cc:844] Serialized 22 byte log entry
0807 19:49:18.627523 (+ 296us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:18.627621 (+ 98us) write_op.cc:454] Released schema lock
0807 19:49:18.627662 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:18.628149 (+ 487us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:18.628197 (+ 48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":317549,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":282157,"apply.run_cpu_time_us":2362,"apply.run_wall_time_us":2363,"num_ops":1,"prepare.queue_time_us":13718,"prepare.run_cpu_time_us":2968,"prepare.run_wall_time_us":9906,"replication_time_us":7899,"spinlock_wait_cycles":64768,"thread_start_us":82313,"threads_started":2}]]}
I20260807 19:49:19.370591 10508 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:19.392160 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:19.403070 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 455556us
W20260807 19:49:19.393908 8788 consensus_meta.cc:287] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.531s user 0.011s sys 0.000s
I20260807 19:49:19.386730 10601 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.382073 10660 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:19.427441 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:19.402292 10508 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:19.448279 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 984) took 1091 ms. Trace:
I20260807 19:49:19.434226 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.448827 8911 rpcz_store.cc:276] 0807 19:49:18.357128 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.459825 (+102697us) service_pool.cc:224] Handling call
0807 19:49:18.460176 (+ 351us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:18.460584 (+ 408us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:18.460609 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:18.461062 (+ 453us) log.cc:844] Serialized 26 byte log entry
0807 19:49:18.861481 (+400419us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:18.992862 (+131381us) log.cc:844] Serialized 14 byte log entry
0807 19:49:19.009297 (+ 16435us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:19.009376 (+ 79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:19.448004 (+438628us) raft_consensus.cc:1685] finished
0807 19:49:19.448029 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:19.448205 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7285,"prepare.queue_time_us":332247,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":1195,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":20038,"thread_start_us":497362,"threads_started":2,"wal-append.queue_time_us":538380}
I20260807 19:49:19.450387 8908 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:19.451120 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.382000 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:19.413175 10702 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:19.465436 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 103570us
I20260807 19:49:19.467836 10670 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:19.468326 10670 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:19.468755 10670 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:19.439815 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:19.411314 10358 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:19.472442 8908 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:19.490513 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:19.492118 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:19.496935 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:19.497705 8787 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:19.523519 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:19.422956 8788 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:19.527478 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:19.474248 10508 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:19.532861 8780 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:19.524304 8787 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:19.474804 10358 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:19.554186 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 67972us
I20260807 19:49:19.523406 10556 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:19.546267 10751 log.cc:927] Time spent T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.098s user 0.001s sys 0.000s
I20260807 19:49:19.400602 10595 consensus_queue.cc:237] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:19.570990 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:19.448760 10756 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
W20260807 19:49:19.563704 10647 consensus_meta.cc:287] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.519s user 0.004s sys 0.008s
W20260807 19:49:19.559494 10703 consensus_meta.cc:287] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.777s user 0.014s sys 0.000s
I20260807 19:49:19.551981 10358 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:19.598850 8780 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:19.560683 10688 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:19.563045 10667 consensus_meta.cc:287] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.577s user 0.012s sys 0.003s
I20260807 19:49:19.601888 8780 pending_rounds.cc:85] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:19.609010 8780 pending_rounds.cc:107] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:19.609309 8780 raft_consensus.cc:2889] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:19.593324 10708 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:19.606580 8903 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s user 0.018s sys 0.019s
W20260807 19:49:19.613072 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 358794us
W20260807 19:49:19.629114 8903 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s user 0.019s sys 0.019s
I20260807 19:49:19.618198 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 988) took 1226 ms. Trace:
W20260807 19:49:19.619009 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 349463us
I20260807 19:49:19.591360 10411 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:19.594694 8904 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:19.622659 8780 raft_consensus.cc:3055] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:19.578138 10716 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260807 19:49:19.623930 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 977) took 1268 ms. Trace:
I20260807 19:49:19.560446 10634 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260807 19:49:19.630092 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 986) took 1272 ms. Trace:
I20260807 19:49:19.630685 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:19.592661 10556 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.601228 10647 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:19.610625 10688 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.175480 10780 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:20.038462 10702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s user 0.007s sys 0.012s
W20260807 19:49:20.227602 10702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s user 0.015s sys 0.014s
I20260807 19:49:20.227604 10780 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:19.911927 10675 consensus_meta.cc:287] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.681s user 0.009s sys 0.000s
W20260807 19:49:20.230715 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 600086us
I20260807 19:49:20.230909 8910 raft_consensus.cc:2368] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:19.663105 10770 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.218887 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 985) took 1861 ms. Trace:
W20260807 19:49:19.664016 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:19.629757 8913 rpcz_store.cc:276] 0807 19:49:18.391961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.503986 (+112025us) service_pool.cc:224] Handling call
0807 19:49:18.515477 (+ 11491us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:18.518787 (+ 3310us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:18.518822 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:18.533798 (+ 14976us) log.cc:844] Serialized 26 byte log entry
0807 19:49:18.928307 (+394509us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:19.169957 (+241650us) log.cc:844] Serialized 14 byte log entry
0807 19:49:19.612749 (+442792us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:19.612817 (+ 68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:19.612857 (+ 40us) raft_consensus.cc:1685] finished
0807 19:49:19.612870 (+ 13us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:19.618127 (+ 5257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96,"prepare.queue_time_us":265586,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":27,"raft.queue_time_us":187263,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":11859,"thread_start_us":1025892,"threads_started":3,"wal-append.queue_time_us":574808}
W20260807 19:49:19.883917 10578 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s user 0.024s sys 0.058s
W20260807 19:49:20.085652 10334 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s user 0.010s sys 0.048s
W20260807 19:49:20.234936 10578 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.876s user 0.024s sys 0.086s
W20260807 19:49:20.194438 10392 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.836s user 0.017s sys 0.009s
I20260807 19:49:20.232100 8901 rpcz_store.cc:276] 0807 19:49:18.357287 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.484183 (+126896us) service_pool.cc:224] Handling call
0807 19:49:18.484562 (+ 379us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:18.484895 (+ 333us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:18.484921 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:18.491036 (+ 6115us) log.cc:844] Serialized 26 byte log entry
0807 19:49:18.617081 (+126045us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:19.225479 (+608398us) log.cc:844] Serialized 14 byte log entry
0807 19:49:20.218444 (+992965us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:20.218547 (+ 103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:20.218606 (+ 59us) raft_consensus.cc:1685] finished
0807 19:49:20.218619 (+ 13us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:20.218795 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":576325,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":128753,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":6099,"thread_start_us":1088409,"threads_started":3,"wal-append.queue_time_us":308064}
I20260807 19:49:19.771082 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:49:20.235810 10392 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.877s user 0.017s sys 0.012s
I20260807 19:49:20.229868 8906 rpcz_store.cc:276] 0807 19:49:18.355388 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.355509 (+ 121us) service_pool.cc:224] Handling call
0807 19:49:18.355828 (+ 319us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:18.361939 (+ 6111us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:18.361971 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:18.377797 (+ 15826us) log.cc:844] Serialized 26 byte log entry
0807 19:49:18.516463 (+138666us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b1800211000. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a32c1 00007f687b5bfdef
0807 19:49:18.936708 (+420245us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:19.162633 (+225925us) log.cc:844] Serialized 14 byte log entry
0807 19:49:19.612484 (+449851us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:19.612572 (+ 88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:19.612623 (+ 51us) raft_consensus.cc:1685] finished
0807 19:49:19.623693 (+ 11070us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:19.623852 (+ 159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7136,"prepare.queue_time_us":257388,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":191117,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":17223,"spinlock_wait_cycles":15986560,"thread_start_us":1067229,"threads_started":3,"wal-append.queue_time_us":658975}
I20260807 19:49:19.798648 8856 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:19.617755 10667 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:20.237133 10334 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.780s user 0.010s sys 0.055s
I20260807 19:49:19.877012 10765 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.235108 10702 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:20.238519 10765 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.236832 10770 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.239157 10688 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:20.233695 8780 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:20.241250 8780 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:20.236495 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 465484us
I20260807 19:49:20.242285 8780 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:19.874235 10411 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.121340 10708 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.245033 10411 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.226735 8903 rpcz_store.cc:276] 0807 19:49:18.357459 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.486619 (+129160us) service_pool.cc:224] Handling call
0807 19:49:18.486968 (+ 349us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:18.492113 (+ 5145us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:18.492144 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:18.494629 (+ 2485us) log.cc:844] Serialized 26 byte log entry
0807 19:49:18.989351 (+494722us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:19.629534 (+640183us) log.cc:844] Serialized 14 byte log entry
0807 19:49:19.629759 (+ 225us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:19.629837 (+ 78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:19.629880 (+ 43us) raft_consensus.cc:1685] finished
0807 19:49:19.629892 (+ 12us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:19.630036 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":204859,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":5423,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":2397,"thread_start_us":1110744,"threads_started":2,"wal-append.queue_time_us":705399}
I20260807 19:49:20.247922 10411 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:19.989567 10727 consensus_meta.cc:287] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.761s user 0.009s sys 0.000s
I20260807 19:49:20.250905 10727 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.252228 8778 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:20.252596 10779 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:20.254320 10727 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:19.853544 10734 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:20.249316 10771 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:20.255335 10771 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:19.912963 10408 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.258327 10734 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:19.898398 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:20.261031 10734 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:19.879766 10700 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.262369 10700 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.265344 10700 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:20.266924 10556 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.226460 8915 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:20.230013 10675 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:19.990716 10758 log.cc:927] Time spent T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.222s user 0.000s sys 0.000s
I20260807 19:49:20.253739 10779 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.233237 8783 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:20.296128 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.256640 10408 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.302814 9021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 938) took 1801 ms. Trace:
I20260807 19:49:20.303282 9021 rpcz_store.cc:276] 0807 19:49:18.502237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.326836 (+824599us) service_pool.cc:224] Handling call
0807 19:49:20.302023 (+975187us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:20.302256 (+ 233us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:20.302479 (+ 223us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:20.302503 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:20.302543 (+ 40us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:20.302567 (+ 24us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:20.302626 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:20.302753 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34703616}
W20260807 19:49:20.291239 10670 consensus_meta.cc:287] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.822s user 0.009s sys 0.003s
I20260807 19:49:20.241245 10770 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:20.313635 10772 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.436s user 0.000s sys 0.001s
I20260807 19:49:20.293701 10727 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:20.333110 10727 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.330826 10670 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:20.338713 10775 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.440s user 0.000s sys 0.000s
I20260807 19:49:20.240126 10688 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:20.339556 10688 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:20.268039 10556 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.345904 10408 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.351011 10670 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:20.344123 10601 consensus_meta.cc:287] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.897s user 0.010s sys 0.001s
I20260807 19:49:20.346896 10727 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.340176 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 991) took 1273 ms. Trace:
W20260807 19:49:20.360003 10549 consensus_meta.cc:287] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 1.289s user 0.005s sys 0.003s
I20260807 19:49:20.362437 8902 rpcz_store.cc:276] 0807 19:49:19.066649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.074864 (+ 8215us) service_pool.cc:224] Handling call
0807 19:49:19.075206 (+ 342us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:19.082953 (+ 7747us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:19.082984 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:19.094668 (+ 11684us) log.cc:844] Serialized 26 byte log entry
0807 19:49:19.390722 (+296054us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:19.390763 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:19.390967 (+ 204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:20.339911 (+948944us) raft_consensus.cc:1685] finished
0807 19:49:20.339933 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:20.340107 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8352,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":14705,"thread_start_us":254591,"threads_started":1,"wal-append.queue_time_us":509675}
W20260807 19:49:20.315439 10683 consensus_meta.cc:287] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.047s user 0.010s sys 0.000s
I20260807 19:49:20.351365 10778 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.310050 10737 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:20.341442 10688 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:20.345201 10556 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.394429 8780 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:20.395174 8780 pending_rounds.cc:85] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:20.395417 8780 pending_rounds.cc:107] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:20.395680 8780 raft_consensus.cc:2889] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:20.370599 10408 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.402933 8970 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:20.406586 8970 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:20.407232 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:20.413312 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 614710us
W20260807 19:49:20.397583 10673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s user 0.022s sys 0.024s
I20260807 19:49:20.414274 8856 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:20.384749 10729 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.397966 10765 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:20.428999 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:20.368278 10787 consensus_queue.cc:1059] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260807 19:49:20.374389 10683 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.373451 10549 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:20.405351 10781 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:20.429548 10673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s user 0.022s sys 0.024s
I20260807 19:49:20.416210 10580 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:20.390496 10737 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:20.432317 10705 consensus_meta.cc:287] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.260s user 0.013s sys 0.000s
I20260807 19:49:20.430092 10689 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:20.434885 10667 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.383394 10778 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:20.457458 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 223457us
I20260807 19:49:20.413546 10408 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.460359 8780 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:20.462010 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.463268 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.463291 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.464704 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.465837 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.466670 8789 raft_consensus.cc:2410] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:20.467448 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.464702 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:20.468595 8779 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:20.478086 8779 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:20.479133 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:20.493445 8740 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items
I20260807 19:49:20.440972 10785 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:20.480304 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.500921 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.451378 10594 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:20.505887 8787 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:20.507911 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.508631 8783 raft_consensus.cc:2376] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:20.495143 10702 raft_consensus.cc:697] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:20.495280 10667 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:20.453091 10705 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.441078 10729 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.499246 9888 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:20.505818 10785 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:20.545318 10358 consensus_meta.cc:287] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.906s user 0.005s sys 0.006s
I20260807 19:49:20.553593 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:20.555078 8911 raft_consensus.cc:2393] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:20.554773 10819 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:20.530939 10594 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.536993 10675 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:20.589414 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 175199us
I20260807 19:49:20.555709 10683 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:20.554742 10674 consensus_meta.cc:287] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.324s user 0.006s sys 0.006s
W20260807 19:49:20.550424 10804 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260807 19:49:20.562513 10556 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.493861 8740 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 941)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 939)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 929)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 935)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 957)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 956)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 954)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
I20260807 19:49:20.560973 10702 consensus_queue.cc:237] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:20.553371 10508 consensus_meta.cc:287] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.022s user 0.010s sys 0.000s
I20260807 19:49:20.562170 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:20.558190 10771 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:20.560858 10695 raft_consensus.cc:1068] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Unable to promote non-voter 39975243456d4452b4a4870ac7ebafc5: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }.
Pending config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:20.585520 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:20.607244 8902 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:20.449844 10781 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:20.584760 10358 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.609110 10512 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:20.610410 10358 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:20.611572 9019 raft_consensus.cc:2393] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:20.612119 9019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 740) took 2229 ms (client timeout 1816 ms). Trace:
I20260807 19:49:20.573632 10819 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:20.542502 10595 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:20.603724 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:20.626156 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:20.563903 10667 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.629802 10595 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.662564 10595 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.639779 10779 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.665578 10737 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.642212 10549 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:20.643921 8902 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:20.648469 10662 consensus_meta.cc:287] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.302s user 0.007s sys 0.008s
I20260807 19:49:20.613427 10508 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.606415 10674 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.654085 10781 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:20.589066 10765 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260807 19:49:20.655577 10408 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.657483 10729 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:20.658185 10785 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.687176 8908 raft_consensus.cc:2393] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:20.663100 8740 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
I20260807 19:49:20.675518 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:20.628401 10819 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:20.630673 10683 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
W20260807 19:49:20.612550 9019 rpcz_store.cc:269] 0807 19:49:18.382275 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.382408 (+ 133us) service_pool.cc:224] Handling call
0807 19:49:20.553404 (+2170996us) spinlock_profiling.cc:243] Waited 86.5 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:20.612012 (+ 58608us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198973824}
I20260807 19:49:20.677819 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:20.679064 10792 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.085s user 0.000s sys 0.001s
I20260807 19:49:20.698205 8783 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:20.689391 8902 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
W20260807 19:49:20.701054 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 74204.5us
W20260807 19:49:20.709858 10808 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.104s user 0.000s sys 0.001s
I20260807 19:49:20.710228 9018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 940) took 2203 ms. Trace:
I20260807 19:49:20.676018 10662 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.702246 10667 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:20.715806 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 124889us
I20260807 19:49:20.722008 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:20.722676 8902 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:20.724066 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:20.724619 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:20.726688 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:20.729054 8903 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:20.718734 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:20.730927 8903 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:20.736771 8783 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:20.738811 10334 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.676752 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 942)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 963)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
W20260807 19:49:20.740960 8849 leader_election.cc:336] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.535s (SENT)
I20260807 19:49:20.808231 8849 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:20.748596 8972 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:20.810503 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 62046.6us
I20260807 19:49:20.812042 10829 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:20.812566 10830 log.cc:927] Time spent T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.101s user 0.001s sys 0.000s
I20260807 19:49:20.812827 10829 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:20.755425 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:20.815407 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:20.811070 10779 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:20.809062 10674 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.759905 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:20.819761 8902 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:20.808629 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 947)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 936)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
I20260807 19:49:20.808868 10334 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.815531 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:20.811367 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:20.817348 8911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1003) took 1367 ms (client timeout 1623 ms). Trace:
I20260807 19:49:20.756641 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:20.757411 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:20.811688 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 70793.8us
I20260807 19:49:20.757050 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:20.967674 8910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 992) took 1889 ms (client timeout 2183 ms). Trace:
I20260807 19:49:21.006237 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:20.785315 10508 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.270130 8783 raft_consensus.cc:2376] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:21.014791 8908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1005) took 1529 ms. Trace:
I20260807 19:49:21.271339 8908 rpcz_store.cc:276] 0807 19:49:19.485163 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.533743 (+ 48580us) service_pool.cc:224] Handling call
0807 19:49:20.561973 (+1028230us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b58001e4b98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.014513 (+452540us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b54002fce54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:21.014710 (+ 197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198750336}
W20260807 19:49:21.054028 10688 consensus_meta.cc:287] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.637s user 0.012s sys 0.000s
I20260807 19:49:21.273118 10688 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.710666 9018 rpcz_store.cc:276] 0807 19:49:18.506781 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.329418 (+822637us) service_pool.cc:224] Handling call
0807 19:49:19.339065 (+ 9647us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:19.339609 (+ 544us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:19.339661 (+ 52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:19.340311 (+ 650us) log.cc:844] Serialized 26 byte log entry
0807 19:49:19.852613 (+512302us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:20.588330 (+735717us) log.cc:844] Serialized 14 byte log entry
0807 19:49:20.659087 (+ 70757us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:20.659183 (+ 96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:20.707657 (+ 48474us) raft_consensus.cc:1685] finished
0807 19:49:20.709958 (+ 2301us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:20.710134 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26912,"prepare.queue_time_us":446869,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":44052,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":267,"thread_start_us":696825,"threads_started":2,"wal-append.queue_time_us":1197449}
W20260807 19:49:21.282325 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 418553us
W20260807 19:49:20.720814 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 182745us
I20260807 19:49:20.799722 8906 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:21.278964 8911 rpcz_store.cc:269] 0807 19:49:19.450073 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.450857 (+ 784us) service_pool.cc:224] Handling call
0807 19:49:20.817162 (+1366305us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004614d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:20.817289 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536135936}
I20260807 19:49:21.280251 8740 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
I20260807 19:49:21.020859 10702 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:20.911795 9015 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:21.276894 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.295959 8790 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:20.977159 8630 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:21.130357 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1045) took 1635 ms. Trace:
I20260807 19:49:21.298520 8778 rpcz_store.cc:276] 0807 19:49:19.495078 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.495244 (+ 166us) service_pool.cc:224] Handling call
0807 19:49:19.495603 (+ 359us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:20.252648 (+757045us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:21.130000 (+877352us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540008c0d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:21.130245 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739644160}
W20260807 19:49:21.137288 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:21.300287 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:20.971187 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.929011 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.294086 8740 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 966)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
I20260807 19:49:21.301151 8787 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260807 19:49:21.278335 8910 rpcz_store.cc:269] 0807 19:49:19.078437 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.079285 (+ 848us) service_pool.cc:224] Handling call
0807 19:49:20.967452 (+1888167us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400360bd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:20.967605 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636963072}
I20260807 19:49:20.962337 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:20.977382 10854 log.cc:927] Time spent T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.196s user 0.001s sys 0.000s
W20260807 19:49:21.108137 10794 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.652s user 0.000s sys 0.001s
I20260807 19:49:20.967020 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:20.972149 10816 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.417s user 0.002s sys 0.000s
I20260807 19:49:21.308020 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.848042 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.308983 8779 raft_consensus.cc:2410] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260807 19:49:21.307297 8849 leader_election.cc:336] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.623s (SENT)
I20260807 19:49:21.310431 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.020233 10820 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:21.018442 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:21.287484 10730 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s user 0.007s sys 0.008s
I20260807 19:49:21.131294 10780 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:21.283885 9020 raft_consensus.cc:2393] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:21.290041 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 930)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 911)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 948)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 944)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 934)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 907)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 958)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 925)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 966)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
I20260807 19:49:21.285924 10702 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:20.818822 10358 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.493s
I20260807 19:49:21.294437 10782 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:20.854060 10834 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.131s user 0.000s sys 0.000s
W20260807 19:49:20.931401 10847 log.cc:927] Time spent T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.147s user 0.001s sys 0.000s
W20260807 19:49:21.010030 10799 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.465s user 0.001s sys 0.001s
W20260807 19:49:21.121940 10824 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.568s user 0.002s sys 0.000s
I20260807 19:49:20.757783 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.003885 10819 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:21.304414 9015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 739) took 2922 ms (client timeout 1752 ms). Trace:
I20260807 19:49:21.311271 8787 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.306349 8780 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
W20260807 19:49:21.311167 8849 leader_election.cc:336] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.651s (SENT)
I20260807 19:49:21.310294 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:20.997663 10594 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:20.890944 10662 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.018088 10770 raft_consensus.cc:3055] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:21.126125 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:21.288033 10334 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:20.963924 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:21.018733 10810 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.088064 10339 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:20.837699 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.274945 9023 raft_consensus.cc:2393] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:21.347596 9019 raft_consensus.cc:2393] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:21.348024 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.348834 10819 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.349220 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.346500 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1075)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1076)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
I20260807 19:49:21.016502 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:21.317431 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.283092 10705 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.282418 10683 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:21.295572 10691 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:21.283792 10778 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.283649 10686 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.317379 10820 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.321062 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.342756 10770 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:21.360882 8779 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.321565 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.362193 8790 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:21.321978 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.280738 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.295277 8776 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
W20260807 19:49:21.314906 10730 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s user 0.007s sys 0.008s
I20260807 19:49:21.327225 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.356196 9008 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.021423 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1048) took 1493 ms. Trace:
I20260807 19:49:21.332489 9027 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.318172 9022 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.337185 8789 raft_consensus.cc:2410] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.369362 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.337518 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:21.338500 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 451295us
W20260807 19:49:21.334913 9015 rpcz_store.cc:269] 0807 19:49:18.381972 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.382652 (+ 680us) service_pool.cc:224] Handling call
0807 19:49:19.267718 (+885066us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.304129 (+2036411us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54002940d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:21.304252 (+ 123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3607243904}
I20260807 19:49:21.341558 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.326601 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:21.344178 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60932us
I20260807 19:49:21.342597 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:21.353392 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.353763 10810 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.353945 10339 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:21.355367 10556 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.357599 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.378198 9023 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:21.279385 9017 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:21.359306 10683 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:21.363472 9021 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:21.353338 10594 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.365607 9020 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.360581 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 962)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1077)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1075)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
I20260807 19:49:21.358489 9893 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:21.335862 10782 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.372597 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.372653 8787 raft_consensus.cc:2376] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
W20260807 19:49:21.373371 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 62235.1us
I20260807 19:49:21.376657 8972 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.393625 8789 raft_consensus.cc:2393] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:21.393828 8849 leader_election.cc:336] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.715s (SENT)
I20260807 19:49:21.379582 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.394644 9017 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.395586 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.396548 8787 raft_consensus.cc:2410] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.383545 8970 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:21.370111 10358 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.370424 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.401355 8972 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.398874 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.402475 9021 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.390545 9023 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.374185 10674 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:21.378759 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.397682 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:21.406579 8849 leader_election.cc:336] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.734s (SENT)
I20260807 19:49:21.400756 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.407660 8787 raft_consensus.cc:2410] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.408581 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.409099 8787 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.409746 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.410259 8787 raft_consensus.cc:2410] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.410822 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.411203 8787 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:21.407044 8849 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.411819 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.402576 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.406893 10692 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:21.404038 10358 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.407050 10705 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:21.393664 10594 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.399757 9018 raft_consensus.cc:2393] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:21.423605 9023 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:21.429016 9021 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:21.382138 10556 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.398653 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.428313 8856 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.412299 10622 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:21.408735 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 95630.6us
I20260807 19:49:21.438431 10785 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:21.437672 8972 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.440443 9020 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:21.408114 9019 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.439448 10686 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:21.442123 10700 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:21.415977 10392 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:21.443652 8780 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.447710 9017 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:21.442576 10700 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:21.449931 8849 leader_election.cc:336] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.793s (SENT)
I20260807 19:49:21.452755 8787 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.421502 10705 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:21.420943 10408 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:21.416286 10688 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:21.457808 8849 leader_election.cc:336] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.872s (SENT)
I20260807 19:49:21.469722 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.458076 8856 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:21.458348 10867 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.459841 10512 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260807 19:49:21.464289 10358 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.464968 10622 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:21.453263 10392 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:21.388839 10770 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.475080 8972 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.445498 10702 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.462409 10730 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:21.456430 10700 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.440220 10810 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:21.480286 8849 leader_election.cc:336] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.914s (SENT)
I20260807 19:49:21.477592 10770 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.477388 10785 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:21.459889 10683 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:21.508455 10647 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.508998 10364 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:21.509276 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.510033 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910) took 1043 ms. Trace:
I20260807 19:49:21.510391 8783 rpcz_store.cc:276] 0807 19:49:20.466431 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.508934 (+1042503us) service_pool.cc:224] Handling call
0807 19:49:21.509969 (+ 1035us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.510902 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927) took 1037 ms. Trace:
I20260807 19:49:21.511299 8787 rpcz_store.cc:276] 0807 19:49:20.476523 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.411601 (+935078us) service_pool.cc:224] Handling call
0807 19:49:21.510664 (+ 99063us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b540009ef54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:21.510837 (+ 173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87752576}
I20260807 19:49:21.481067 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.513149 10647 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:21.480611 10730 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:21.513881 10730 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.508378 10594 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.515770 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.516446 8790 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.478508 10392 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.519047 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.519280 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.481997 8856 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:21.513239 8849 leader_election.cc:336] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.984s (SENT)
I20260807 19:49:21.519726 8789 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
W20260807 19:49:21.520747 8849 leader_election.cc:336] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.076s (SENT)
I20260807 19:49:21.521195 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.512778 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.456786 10686 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.521548 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937) took 1042 ms. Trace:
I20260807 19:49:21.523087 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
W20260807 19:49:21.522284 8849 leader_election.cc:336] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.133s (SENT)
I20260807 19:49:21.523195 8787 raft_consensus.cc:2376] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:21.519953 8779 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.523290 8789 rpcz_store.cc:276] 0807 19:49:20.478980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.518768 (+1039788us) service_pool.cc:224] Handling call
0807 19:49:21.521486 (+ 2718us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.511425 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.524575 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.524566 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.525441 8789 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:21.525756 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943) took 1045 ms. Trace:
I20260807 19:49:21.525961 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066) took 1024 ms. Trace:
I20260807 19:49:21.526057 8849 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.526635 8789 rpcz_store.cc:276] 0807 19:49:20.502039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.524244 (+1022205us) service_pool.cc:224] Handling call
0807 19:49:21.525911 (+ 1667us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42496}
I20260807 19:49:21.527016 8790 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:21.525823 8856 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.361332 8740 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 961)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 910)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 916)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 949)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 908)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 913)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 919)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 955)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 915)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 927)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 943)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 937)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 964)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1066)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 967)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1077)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1075)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
I20260807 19:49:21.530632 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 921) took 1058 ms. Trace:
I20260807 19:49:21.531247 8783 rpcz_store.cc:276] 0807 19:49:20.471884 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.511106 (+1039222us) service_pool.cc:224] Handling call
0807 19:49:21.530565 (+ 19459us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.530742 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.531968 8789 raft_consensus.cc:2376] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:21.532471 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 945) took 1051 ms. Trace:
I20260807 19:49:21.532891 8780 rpcz_store.cc:276] 0807 19:49:20.481314 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.341276 (+859962us) service_pool.cc:224] Handling call
0807 19:49:21.532262 (+190986us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b540008cfd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:21.532414 (+ 152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164732672}
I20260807 19:49:21.532465 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 922) took 1060 ms. Trace:
I20260807 19:49:21.533928 8787 rpcz_store.cc:276] 0807 19:49:20.480775 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.512187 (+1031412us) service_pool.cc:224] Handling call
0807 19:49:21.525702 (+ 13515us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.534166 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.534504 8789 rpcz_store.cc:276] 0807 19:49:20.472139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.530485 (+1058346us) service_pool.cc:224] Handling call
0807 19:49:21.532411 (+ 1926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.536473 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.595432 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.671991 10716 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.542706 10683 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:21.507249 10626 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:21.366523 8788 rpcz_store.cc:276] 0807 19:49:19.528420 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:19.539395 (+ 10975us) service_pool.cc:224] Handling call
0807 19:49:20.551230 (+1011835us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:20.551488 (+ 258us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:20.551725 (+ 237us) raft_consensus.cc:1534] Early marking committed up to index 2
0807 19:49:20.552043 (+ 318us) log.cc:844] Serialized 14 byte log entry
0807 19:49:21.021078 (+469035us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:21.021136 (+ 58us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:21.021179 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:21.021245 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:21.021366 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":265124,"prepare.queue_time_us":591,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":264567,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"spinlock_wait_cycles":6031744,"thread_start_us":197896,"threads_started":1,"wal-append.queue_time_us":373943}
I20260807 19:49:21.673993 10626 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:21.674171 10692 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:21.674437 10626 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.674820 10683 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:21.675221 10683 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.674661 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.675657 8775 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 2. (term mismatch)
I20260807 19:49:21.674728 10692 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.677429 10700 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.676108 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 901) took 1213 ms. Trace:
I20260807 19:49:21.679242 10692 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.679943 8788 rpcz_store.cc:276] 0807 19:49:20.462645 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.674342 (+1211697us) service_pool.cc:224] Handling call
0807 19:49:21.676045 (+ 1703us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.680682 10716 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.681049 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.681807 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 953) took 1198 ms. Trace:
I20260807 19:49:21.682152 8788 rpcz_store.cc:276] 0807 19:49:20.483488 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.680732 (+1197244us) service_pool.cc:224] Handling call
0807 19:49:21.681748 (+ 1016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.682893 10716 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.683146 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.676056 8775 pending_rounds.cc:85] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:21.684415 8775 pending_rounds.cc:107] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.2
I20260807 19:49:21.684554 10820 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.625864 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 917) took 1157 ms. Trace:
I20260807 19:49:21.686081 8779 rpcz_store.cc:276] 0807 19:49:20.468262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.533229 (+1064967us) service_pool.cc:224] Handling call
0807 19:49:21.625662 (+ 92433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.687013 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.687160 10686 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.687691 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 959) took 1026 ms. Trace:
I20260807 19:49:21.472035 10408 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:21.474551 10867 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.684731 8775 raft_consensus.cc:2917] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Aborting config change with OpId 1.2: Aborted: Op aborted by new leader
I20260807 19:49:21.688083 8778 raft_consensus.cc:2410] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.688462 10392 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.689287 8775 raft_consensus.cc:2889] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:21.541306 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:21.689937 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:53502 (request call id 4) took 6626 ms. Trace:
W20260807 19:49:21.638206 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.644780 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.572718 10819 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260807 19:49:21.683980 9019 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:21.691351 10819 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:21.690372 8775 rpcz_store.cc:276] 0807 19:49:15.064052 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:15.064261 (+ 209us) service_pool.cc:224] Handling call
0807 19:49:15.096064 (+ 31803us) log.cc:844] Serialized 499 byte log entry
0807 19:49:15.455236 (+359172us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b1800074e20. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6410e1 00007f687b57da71 00007f687b55e28a 00007f687b565133 00007f687b5650b9 00007f687b565049 00007f687b565011 00007f687b5641dd 00000000003aa554 00000000003aa399
0807 19:49:21.689846 (+6234610us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29206,"raft.queue_time_us":56194,"raft.run_cpu_time_us":7101,"raft.run_wall_time_us":218371,"spinlock_wait_cycles":2534144,"thread_start_us":176306,"threads_started":2,"wal-append.queue_time_us":180820,"wal-append.run_cpu_time_us":2779,"wal-append.run_wall_time_us":1100012}
I20260807 19:49:21.532910 10647 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:21.541887 10674 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:21.692798 10674 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:21.693207 10674 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:21.689971 10867 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.486420 10358 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.146s
I20260807 19:49:21.695142 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 946) took 1213 ms. Trace:
I20260807 19:49:21.695490 8783 rpcz_store.cc:276] 0807 19:49:20.481591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.531895 (+1050304us) service_pool.cc:224] Handling call
0807 19:49:21.695080 (+163185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.696532 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.697216 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 960) took 1035 ms. Trace:
I20260807 19:49:21.688011 8779 rpcz_store.cc:276] 0807 19:49:20.661417 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.686771 (+1025354us) service_pool.cc:224] Handling call
0807 19:49:21.687634 (+ 863us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.698037 10787 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:21.691727 10819 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:21.698582 10787 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.699456 10819 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:21.700902 10819 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:21.701109 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1039) took 3007 ms. Trace:
I20260807 19:49:21.701542 8775 rpcz_store.cc:276] 0807 19:49:18.693952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.708394 (+ 14442us) service_pool.cc:224] Handling call
0807 19:49:18.733775 (+ 25381us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:21.692705 (+2958930us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:21.700849 (+ 8144us) spinlock_profiling.cc:243] Waited 2.93 s on lock 0x7b540008ff54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:21.701042 (+ 193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6748820224}
I20260807 19:49:21.476080 10765 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260807 19:49:21.664794 8856 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.667328 10730 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.705222 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.706117 8787 raft_consensus.cc:2410] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.706712 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 952) took 1223 ms. Trace:
I20260807 19:49:21.707024 8787 rpcz_store.cc:276] 0807 19:49:20.483210 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.534946 (+1051736us) service_pool.cc:224] Handling call
0807 19:49:21.705071 (+170125us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.706650 (+ 1579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3288448}
I20260807 19:49:21.707319 10730 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.708634 10339 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:21.709437 10866 consensus_queue.cc:1059] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:21.710253 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.709297 10778 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:21.505385 10705 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:21.504930 8972 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:21.670164 8849 leader_election.cc:336] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.138s (SENT)
I20260807 19:49:21.667959 9022 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
W20260807 19:49:21.723904 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.692296 10408 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:21.698539 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.478581 10662 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260807 19:49:21.528275 10622 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.684072 8788 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.697547 8783 rpcz_store.cc:276] 0807 19:49:20.661727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.696221 (+1034494us) service_pool.cc:224] Handling call
0807 19:49:21.697167 (+ 946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.703773 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:21.708379 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 144186us
I20260807 19:49:21.541430 10831 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:21.668815 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 147460us
I20260807 19:49:21.742607 8856 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.708060 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.712184 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 928) took 1235 ms. Trace:
I20260807 19:49:21.743808 8789 rpcz_store.cc:276] 0807 19:49:20.476843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.535339 (+1058496us) service_pool.cc:224] Handling call
0807 19:49:21.710098 (+174759us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.712118 (+ 2020us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11387264}
I20260807 19:49:21.744448 10647 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:21.745018 10647 raft_consensus.cc:697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:21.745924 10647 consensus_queue.cc:237] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.742385 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 905) took 1278 ms. Trace:
I20260807 19:49:21.748929 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.745030 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.748432 10781 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260807 19:49:21.748667 8788 rpcz_store.cc:276] 0807 19:49:20.463604 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.682853 (+1219249us) service_pool.cc:224] Handling call
0807 19:49:21.741426 (+ 58573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.749796 8783 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.669471 10780 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:21.721583 10860 raft_consensus.cc:3055] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:21.753959 10364 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:21.753928 10860 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:21.750878 10785 raft_consensus.cc:3055] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:21.755477 10860 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:21.756177 10785 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:21.756661 10860 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.757203 10785 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.758539 10785 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:21.723320 8849 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.724967 8779 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:21.749527 10512 consensus_queue.cc:1059] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260807 19:49:21.761471 10512 raft_consensus.cc:3055] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:21.761840 10512 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:21.762010 10781 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:21.762972 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 92861.7us
I20260807 19:49:21.762760 10512 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:21.763614 8849 leader_election.cc:336] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.064s (SENT)
I20260807 19:49:21.763998 10512 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:21.760561 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.765409 8788 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
W20260807 19:49:21.766620 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.761644 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:21.770730 8739 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items [suppressed 9 similar messages]
W20260807 19:49:21.726179 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 221368us
I20260807 19:49:21.726534 10358 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:21.771072 8739 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1083)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 978)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1078)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 980)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 981)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 982)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 983)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 985)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1090)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 986)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 988)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1091)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1085)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 966)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1080)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1081)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1087)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 993)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1088)
I20260807 19:49:21.727031 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:21.727700 10683 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.690542 10392 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:21.731257 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:21.726711 10778 raft_consensus.cc:697] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:21.732681 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.733121 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.793983 8911 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:21.787498 8740 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1083)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 978)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1078)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 980)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 981)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 982)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 983)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1096)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 985)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1090)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 986)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 988)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1091)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1085)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1080)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1081)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1087)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 993)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1088)
I20260807 19:49:21.734462 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.722671 10705 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:21.752157 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 931) took 1265 ms. Trace:
I20260807 19:49:21.679293 10700 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:21.739001 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.740445 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 933) took 1262 ms. Trace:
I20260807 19:49:21.809681 8778 rpcz_store.cc:276] 0807 19:49:20.478003 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.378473 (+900470us) service_pool.cc:224] Handling call
0807 19:49:21.725217 (+346744us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540008b1d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:21.740369 (+ 15152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":648233856}
I20260807 19:49:21.722326 10556 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.751842 8787 raft_consensus.cc:2368] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:21.817790 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.743541 8856 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:21.818902 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 75433.6us
I20260807 19:49:21.752848 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 932) took 1275 ms. Trace:
W20260807 19:49:21.766412 8849 leader_election.cc:336] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.192s (SENT)
I20260807 19:49:21.726943 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.776078 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.828411 8779 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:21.776724 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 976)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1083)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 978)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1078)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 980)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 981)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 982)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 983)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1096)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 985)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1090)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 986)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 988)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1091)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1085)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1080)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1081)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1087)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 993)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1088)
I20260807 19:49:21.780517 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.783438 10695 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:21.787209 10787 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:21.788231 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.766426 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.789151 10780 raft_consensus.cc:697] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:21.790555 8972 leader_election.cc:400] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:21.842398 8972 leader_election.cc:403] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:21.798514 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.843389 8780 raft_consensus.cc:2410] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.738098 9018 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:21.725716 10674 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:21.810758 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.815207 8911 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:21.820967 8849 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.848665 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 950) took 1366 ms. Trace:
I20260807 19:49:21.849220 8780 rpcz_store.cc:276] 0807 19:49:20.482667 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.533767 (+1051100us) service_pool.cc:224] Handling call
0807 19:49:21.798364 (+264597us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.848579 (+ 50215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8048896}
I20260807 19:49:21.821895 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.850617 8778 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:21.823872 8740 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 977)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 978)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 996)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1078)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 980)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 981)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 982)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 983)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1096)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 985)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1090)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 986)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 988)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1091)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1085)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1080)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1081)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1087)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 993)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1088)
I20260807 19:49:21.828840 8856 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.820757 8783 rpcz_store.cc:276] 0807 19:49:20.477785 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.740738 (+1262953us) service_pool.cc:224] Handling call
0807 19:49:21.752782 (+ 12044us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.830067 8790 raft_consensus.cc:2410] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:21.807550 10705 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.839349 8789 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.838967 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.812446 9019 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:21.768217 10626 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:21.859251 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 68888.1us
I20260807 19:49:21.855494 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 926) took 1382 ms. Trace:
I20260807 19:49:21.860164 8789 rpcz_store.cc:276] 0807 19:49:20.473231 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.751302 (+1278071us) service_pool.cc:224] Handling call
0807 19:49:21.788065 (+ 36763us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.855434 (+ 67369us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4281216}
I20260807 19:49:21.856532 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 912) took 1389 ms. Trace:
I20260807 19:49:21.830891 10781 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:21.858372 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 951) took 1375 ms. Trace:
I20260807 19:49:21.862911 8779 rpcz_store.cc:276] 0807 19:49:20.482936 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.838624 (+1355688us) service_pool.cc:224] Handling call
0807 19:49:21.858300 (+ 19676us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:21.845980 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.864233 8787 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.808993 8776 rpcz_store.cc:276] 0807 19:49:20.477556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.521486 (+1043930us) service_pool.cc:224] Handling call
0807 19:49:21.742658 (+221172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:21.870978 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:21.862334 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:21.849658 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 83253.8us
I20260807 19:49:21.861503 8790 rpcz_store.cc:276] 0807 19:49:20.466946 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.533057 (+1066111us) service_pool.cc:224] Handling call
0807 19:49:21.780413 (+247356us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c693c 00007f68784c68a6 000000000038d209 00007f6880585940 00007f688064ded4 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:21.856454 (+ 76041us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6432512}
W20260807 19:49:21.871428 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.841461 8775 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:21.870471 8910 raft_consensus.cc:1240] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:21.831638 10683 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:21.872582 8849 leader_election.cc:336] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.201s (SENT)
I20260807 19:49:21.879726 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:21.831126 8739 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 978)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 979)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 965)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 996)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 973)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1078)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 980)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 981)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 982)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 983)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 974)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1096)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 984)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 975)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 985)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1090)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 986)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 987)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 988)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 989)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1091)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1097)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 990)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 991)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1085)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 992)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 994)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 998)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1080)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1081)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1087)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 993)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1088)
I20260807 19:49:21.855997 10831 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:21.862617 10756 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:21.880312 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.851760 8788 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:21.883999 8790 raft_consensus.cc:2376] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:21.885599 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.886401 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.886677 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:21.887187 8908 raft_consensus.cc:2368] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:21.888438 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.889636 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:21.892110 8790 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.887236 8901 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:21.890736 8908 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260807 19:49:21.894291 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:21.886454 8911 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:21.900620 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:21.901396 8903 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:21.902302 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:21.906754 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 86203.9us
I20260807 19:49:21.907642 8856 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:21.914897 8790 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:21.916587 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.921540 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.922446 8902 raft_consensus.cc:2393] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 2: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:21.924500 8903 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:21.930047 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.930830 8903 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:49:21.882287 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 918) took 1410 ms. Trace:
I20260807 19:49:21.917379 8790 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:21.941443 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.942096 8779 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.945183 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.945984 8778 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:21.946471 8911 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:21.947814 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.889505 10683 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:21.872823 9850 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:21.952093 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:21.939214 8903 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:21.949826 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:21.953756 9017 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:21.952741 8911 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:21.940733 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.947824 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:21.957952 8908 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:21.953617 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:21.873687 10778 consensus_queue.cc:237] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:21.993969 8906 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:22.070714 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.961789 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:21.958737 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.095885 8972 leader_election.cc:400] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:22.096212 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 923) took 1623 ms. Trace:
I20260807 19:49:22.030942 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.096668 8790 rpcz_store.cc:276] 0807 19:49:20.472468 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.958438 (+1485970us) service_pool.cc:224] Handling call
0807 19:49:22.096144 (+137706us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.097678 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.098037 8783 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:22.102236 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:22.102887 8915 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:49:22.104384 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.105250 8787 raft_consensus.cc:2393] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:22.106706 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:22.107438 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.114637 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.115643 8790 raft_consensus.cc:2410] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.116590 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.117218 8790 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:22.105026 8780 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:22.120739 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:22.012164 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:22.127852 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:22.127851 8780 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.134132 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.142665 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:22.137862 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:22.156167 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:22.157307 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.158301 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 924) took 1685 ms. Trace:
I20260807 19:49:22.158641 8787 rpcz_store.cc:276] 0807 19:49:20.472728 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.156940 (+1684212us) service_pool.cc:224] Handling call
0807 19:49:22.158233 (+ 1293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.159693 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.160481 8787 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:22.160981 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 940) took 1681 ms. Trace:
I20260807 19:49:22.161278 8787 rpcz_store.cc:276] 0807 19:49:20.479826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.159393 (+1679567us) service_pool.cc:224] Handling call
0807 19:49:22.160924 (+ 1531us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.162256 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.166486 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.167616 8780 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.017716 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.096242 8972 leader_election.cc:403] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260807 19:49:22.174436 8789 raft_consensus.cc:2376] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:21.947962 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.186103 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.186861 8783 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:22.186790 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.952330 10819 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:21.959164 10782 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:22.197772 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.047047 9022 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:22.092799 10787 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:22.018186 10392 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:22.053025 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.045862 10781 raft_consensus.cc:697] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:22.193629 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 97917.1us
I20260807 19:49:22.184605 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.184973 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.204420 8790 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:22.231784 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:22.057243 10780 consensus_queue.cc:237] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:22.257649 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:22.319322 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:22.722867 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:22.300864 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:21.936094 10831 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:22.236584 8789 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:22.216044 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:22.732568 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:22.733111 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:22.734746 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:22.247845 8915 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:22.278983 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:22.723511 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:21.941250 8775 rpcz_store.cc:276] 0807 19:49:20.470985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.724062 (+1253077us) service_pool.cc:224] Handling call
0807 19:49:21.881317 (+157255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.724496 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:22.235947 8780 raft_consensus.cc:2376] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:22.728832 8902 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:22.766620 8972 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:22.767702 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 990) took 1002 ms. Trace:
I20260807 19:49:22.235863 8787 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:22.768189 8780 rpcz_store.cc:276] 0807 19:49:21.765113 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.184668 (+419555us) service_pool.cc:224] Handling call
0807 19:49:22.767616 (+582948us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.769418 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.768769 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1085) took 1015 ms. Trace:
I20260807 19:49:22.770224 8787 rpcz_store.cc:276] 0807 19:49:21.753599 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.197456 (+443857us) service_pool.cc:224] Handling call
0807 19:49:22.768711 (+571255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.770094 8780 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:22.771728 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.772327 8780 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:22.772807 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 992) took 1007 ms. Trace:
I20260807 19:49:22.773115 8780 rpcz_store.cc:276] 0807 19:49:21.765777 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.771458 (+1005681us) service_pool.cc:224] Handling call
0807 19:49:22.772754 (+ 1296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.773590 8805 tablet_copy_service.cc:140] P 8e9ad2b5e2a94d4faff90af735c6d792: Received BeginTabletCopySession request for tablet 0ba47e79caa54c409c53304a66ce0160 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:53512)
I20260807 19:49:22.774093 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.774816 8805 tablet_copy_service.cc:161] P 8e9ad2b5e2a94d4faff90af735c6d792: Beginning new tablet copy session on tablet 0ba47e79caa54c409c53304a66ce0160 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:53512: session id = 39975243456d4452b4a4870ac7ebafc5-0ba47e79caa54c409c53304a66ce0160
I20260807 19:49:22.774996 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 994) took 1008 ms. Trace:
I20260807 19:49:22.775431 8780 rpcz_store.cc:276] 0807 19:49:21.766341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.773818 (+1007477us) service_pool.cc:224] Handling call
0807 19:49:22.774936 (+ 1118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.763422 10893 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:22.778959 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.780397 8780 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.781581 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.782593 8780 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:22.783607 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.163652 10556 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:22.348165 10871 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.164s user 0.000s sys 0.000s
I20260807 19:49:22.191462 10741 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 19:49:22.667455 10884 log.cc:927] Time spent T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.222s user 0.000s sys 0.000s
I20260807 19:49:22.784523 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1092) took 1015 ms. Trace:
I20260807 19:49:22.790406 8911 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
W20260807 19:49:22.626500 8904 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s user 0.007s sys 0.027s
W20260807 19:49:22.719202 10860 consensus_meta.cc:287] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.962s user 0.011s sys 0.000s
W20260807 19:49:22.445981 10865 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.123s user 0.000s sys 0.000s
I20260807 19:49:22.792944 8972 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:22.395012 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:22.199479 10877 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:22.808681 10877 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:22.198715 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 337442us
W20260807 19:49:22.813017 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:22.819677 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.820466 8790 raft_consensus.cc:2376] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:22.820981 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1069) took 1503 ms. Trace:
I20260807 19:49:22.821290 8790 rpcz_store.cc:276] 0807 19:49:21.317145 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.789066 (+1471921us) service_pool.cc:224] Handling call
0807 19:49:22.820916 (+ 31850us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.822310 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.829344 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.830148 8790 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:22.802265 8780 rpcz_store.cc:276] 0807 19:49:21.769429 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.783345 (+1013916us) service_pool.cc:224] Handling call
0807 19:49:22.784459 (+ 1114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.201655 10689 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:22.321969 10674 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:22.839566 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.840543 8778 raft_consensus.cc:2410] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.841012 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 974) took 1176 ms. Trace:
I20260807 19:49:22.841328 8778 rpcz_store.cc:276] 0807 19:49:21.664255 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.839264 (+1175009us) service_pool.cc:224] Handling call
0807 19:49:22.840955 (+ 1691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.235095 10819 raft_consensus.cc:697] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:22.199940 8776 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:22.784277 8805 tablet_copy_source_session.cc:215] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:22.798897 10692 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:22.306227 8788 raft_consensus.cc:2376] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:22.795547 10695 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:22.806988 8904 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s user 0.008s sys 0.029s
I20260807 19:49:22.197275 10683 raft_consensus.cc:697] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:22.300709 10787 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:22.300971 10392 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:22.856166 8790 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:22.859237 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:22.866040 8902 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:49:22.827705 10741 raft_consensus.cc:697] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
W20260807 19:49:22.867733 8908 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s user 0.035s sys 0.047s
W20260807 19:49:22.868511 8908 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s user 0.036s sys 0.047s
W20260807 19:49:22.867929 8968 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.586s user 0.007s sys 0.041s
I20260807 19:49:22.798640 10556 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:22.853593 10695 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:22.853788 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.855346 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 980) took 1090 ms. Trace:
W20260807 19:49:22.882568 10874 log.cc:927] Time spent T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Append to log took a long time: real 0.962s user 0.007s sys 0.014s
I20260807 19:49:22.872301 10695 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:22.884161 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.884598 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.885406 8779 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.885573 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.886780 10392 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:22.887681 9018 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:22.886729 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.889740 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:22.862094 10782 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:22.892470 9019 consensus_meta.cc:287] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.023s user 0.004s sys 0.008s
I20260807 19:49:22.861042 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 983) took 1094 ms. Trace:
I20260807 19:49:22.895040 8789 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:22.895560 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 970) took 1556 ms. Trace:
I20260807 19:49:22.902855 8779 rpcz_store.cc:276] 0807 19:49:21.339112 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.884284 (+1545172us) service_pool.cc:224] Handling call
0807 19:49:22.895487 (+ 11203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.904096 8779 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:22.904475 8779 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:22.870508 8968 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.589s user 0.007s sys 0.041s
W20260807 19:49:22.905512 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 623893us
W20260807 19:49:22.908974 9848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s user 0.090s sys 0.146s
W20260807 19:49:22.910265 9848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s user 0.091s sys 0.146s
I20260807 19:49:22.911433 8789 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:22.913488 8789 pending_rounds.cc:85] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:22.914069 8789 pending_rounds.cc:107] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:22.908974 8779 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:22.914358 8789 raft_consensus.cc:2889] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:22.915274 8779 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:22.915557 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 969) took 1592 ms. Trace:
I20260807 19:49:22.917012 8790 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:22.917718 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 920) took 2446 ms. Trace:
I20260807 19:49:22.918043 8790 rpcz_store.cc:276] 0807 19:49:20.471625 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.912182 (+2440557us) service_pool.cc:224] Handling call
0807 19:49:22.912545 (+ 363us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:22.917425 (+ 4880us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:22.917655 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:22.919667 8849 leader_election.cc:336] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.500s (SENT)
I20260807 19:49:22.920230 8849 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:22.923415 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:22.916474 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:22.931959 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:22.919335 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:22.916035 8789 rpcz_store.cc:276] 0807 19:49:21.323181 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.901932 (+1578751us) service_pool.cc:224] Handling call
0807 19:49:22.902281 (+ 349us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:22.915259 (+ 12978us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:22.915497 (+ 238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.935115 8790 raft_consensus.cc:2368] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:22.939108 8902 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:22.940155 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1072) took 1605 ms. Trace:
I20260807 19:49:22.940696 8790 rpcz_store.cc:276] 0807 19:49:21.334274 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.939174 (+1604900us) service_pool.cc:224] Handling call
0807 19:49:22.939562 (+ 388us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:22.939789 (+ 227us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:22.939817 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:22.939868 (+ 51us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:22.939909 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:22.939968 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:22.940084 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.941461 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:22.948916 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1073) took 1608 ms. Trace:
I20260807 19:49:22.949479 8790 rpcz_store.cc:276] 0807 19:49:21.340418 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.947829 (+1607411us) service_pool.cc:224] Handling call
0807 19:49:22.948271 (+ 442us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:22.948529 (+ 258us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:22.948558 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:22.948607 (+ 49us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:22.948639 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:22.948695 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:22.948839 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:22.957243 10647 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s user 0.000s sys 0.024s
W20260807 19:49:22.929883 8968 leader_election.cc:336] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51024 dropped due to backpressure: service queue is full with 50 items
I20260807 19:49:22.960832 8790 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260807 19:49:22.913434 10705 consensus_meta.cc:287] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.051s user 0.007s sys 0.004s
W20260807 19:49:22.911383 10512 consensus_meta.cc:287] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.137s user 0.009s sys 0.003s
I20260807 19:49:22.864856 10781 consensus_queue.cc:237] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:23.132849 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1074) took 1790 ms. Trace:
I20260807 19:49:23.133477 8790 rpcz_store.cc:276] 0807 19:49:21.342172 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.957184 (+1615012us) service_pool.cc:224] Handling call
0807 19:49:22.957639 (+ 455us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.132502 (+174863us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.132773 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:23.083885 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 291001us
I20260807 19:49:23.140084 8972 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:22.945183 8902 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:22.911242 9019 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
W20260807 19:49:23.142225 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 220657us
W20260807 19:49:23.143213 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 204912us
I20260807 19:49:23.017057 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:23.141886 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:23.147164 8779 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:22.901124 10782 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:22.885289 10741 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:23.166918 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 114118us
I20260807 19:49:23.170332 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:22.900964 8775 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:23.171291 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1080) took 1775 ms. Trace:
I20260807 19:49:23.170431 8789 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:23.805502 8789 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:23.805831 8789 pending_rounds.cc:85] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:23.806077 8789 pending_rounds.cc:107] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:23.806403 8789 raft_consensus.cc:2889] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:23.807240 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 993) took 2041 ms. Trace:
I20260807 19:49:23.807542 8789 rpcz_store.cc:276] 0807 19:49:21.766050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.804456 (+2038406us) service_pool.cc:224] Handling call
0807 19:49:23.804930 (+ 474us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.806961 (+ 2031us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.807167 (+ 206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:23.174211 8849 leader_election.cc:336] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.531s (SENT)
I20260807 19:49:23.812930 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:23.814677 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 640463us
W20260807 19:49:23.815644 8849 leader_election.cc:336] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.565s (SENT)
I20260807 19:49:23.172796 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.171185 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.189734 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1054) took 1246 ms. Trace:
I20260807 19:49:23.863770 8910 rpcz_store.cc:276] 0807 19:49:21.943402 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.768948 (+825546us) service_pool.cc:224] Handling call
0807 19:49:22.769253 (+ 305us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:22.769451 (+ 198us) raft_consensus.cc:1534] Early marking committed up to index 2
0807 19:49:22.932858 (+163407us) log.cc:844] Serialized 14 byte log entry
0807 19:49:23.189365 (+256507us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:23.189429 (+ 64us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:23.189471 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.189539 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.189673 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38591,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":34,"raft.queue_time_us":1475,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":1782,"thread_start_us":382641,"threads_started":2,"wal-append.queue_time_us":281707}
I20260807 19:49:23.169875 8915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1055) took 1208 ms. Trace:
I20260807 19:49:23.867219 8915 rpcz_store.cc:276] 0807 19:49:21.961826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.785362 (+823536us) service_pool.cc:224] Handling call
0807 19:49:22.785779 (+ 417us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:22.786222 (+ 443us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:22.786256 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:22.801363 (+ 15107us) log.cc:844] Serialized 26 byte log entry
0807 19:49:23.084529 (+283166us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.084590 (+ 61us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.084677 (+ 87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:23.169579 (+ 84902us) raft_consensus.cc:1685] finished
0807 19:49:23.169605 (+ 26us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.169783 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":759,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":7693,"thread_start_us":131198,"threads_started":1,"wal-append.queue_time_us":331316}
I20260807 19:49:23.869709 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.870536 8915 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:23.871589 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.174405 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:23.879882 10829 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:23.880580 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 706144us
I20260807 19:49:23.881325 10829 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:23.881904 8968 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:23.173341 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:23.184890 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37380 (request call id 66) took 1447 ms. Trace:
I20260807 19:49:23.180585 10935 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:23.178853 10915 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:23.885105 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:23.338600 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:23.179566 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 133903us
I20260807 19:49:23.888921 10756 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:23.888522 10935 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:23.170056 10392 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 19:49:23.892838 10935 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:23.186474 10556 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:23.172701 10512 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
W20260807 19:49:23.132048 10647 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s user 0.003s sys 0.069s
I20260807 19:49:23.101293 10683 consensus_queue.cc:237] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:22.959079 10688 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:23.130472 10695 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:23.170862 10705 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:23.171947 10782 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:23.208429 10787 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:22.931025 10819 consensus_queue.cc:237] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:23.015019 10860 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:23.148963 10889 consensus_queue.cc:1059] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260807 19:49:22.929028 10896 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:23.126166 10905 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.254s user 0.000s sys 0.001s
I20260807 19:49:23.511264 8779 pending_rounds.cc:85] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:23.969429 8779 pending_rounds.cc:107] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:23.969734 8779 raft_consensus.cc:2889] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:23.970746 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1070) took 2645 ms. Trace:
I20260807 19:49:23.971060 8779 rpcz_store.cc:276] 0807 19:49:21.325400 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.903655 (+1578255us) service_pool.cc:224] Handling call
0807 19:49:22.903974 (+ 319us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.970390 (+1066416us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.970672 (+ 282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:22.876049 8776 rpcz_store.cc:276] 0807 19:49:21.762493 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.888508 (+126015us) service_pool.cc:224] Handling call
0807 19:49:22.852314 (+963806us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:23.522646 8906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 81) took 1791 ms (client timeout 2031 ms). Trace:
I20260807 19:49:23.554524 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1081) took 2149 ms. Trace:
I20260807 19:49:23.589169 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1053) took 1659 ms. Trace:
I20260807 19:49:23.616722 8908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1039) took 1715 ms. Trace:
I20260807 19:49:23.618793 9018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 978) took 1690 ms. Trace:
I20260807 19:49:23.623104 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1026) took 1737 ms. Trace:
I20260807 19:49:23.626740 9015 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:23.629257 8740 connection.cc:256] RPC call timeout handler was delayed by -4.8131322800181806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21280000000000002s for a response
I20260807 19:49:23.973053 8783 rpcz_store.cc:276] 0807 19:49:21.404662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.552472 (+2147810us) service_pool.cc:224] Handling call
0807 19:49:23.553363 (+ 891us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.553830 (+ 467us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.553859 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:23.553988 (+ 129us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.554042 (+ 54us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.554130 (+ 88us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.554274 (+ 144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:23.975013 9015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 979) took 2046 ms. Trace:
I20260807 19:49:23.631687 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1087) took 1877 ms. Trace:
W20260807 19:49:23.637125 8714 auto_rebalancer.cc:644] Failed to schedule move for tablet 0ba47e79caa54c409c53304a66ce0160: Aborted: Op aborted by new leader
W20260807 19:49:23.644917 10880 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 1.294s user 0.011s sys 0.003s
I20260807 19:49:23.653316 10950 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:23.977262 10950 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:23.975850 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:23.979686 8740 connection.cc:256] RPC call timeout handler was delayed by -3.2595084710046649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
I20260807 19:49:23.657608 10626 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:23.689118 10364 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:23.972952 8906 rpcz_store.cc:269] 0807 19:49:21.730290 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.732460 (+ 2170us) service_pool.cc:224] Handling call
0807 19:49:23.521827 (+1789367us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000b3954. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:23.522143 (+ 316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258828032}
I20260807 19:49:23.981307 10364 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:23.981703 10556 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:23.982003 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.983091 8906 raft_consensus.cc:2393] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:23.712828 10778 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:23.983417 10556 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:23.983719 10778 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:23.984139 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.984901 8906 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:23.716010 10781 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:23.973533 8901 rpcz_store.cc:276] 0807 19:49:21.929400 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.636970 (+707570us) service_pool.cc:224] Handling call
0807 19:49:22.637327 (+ 357us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:22.856877 (+219550us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:22.856910 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:22.864934 (+ 8024us) log.cc:844] Serialized 26 byte log entry
0807 19:49:23.050848 (+185914us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.050922 (+ 74us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.588148 (+537226us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54000a4e54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:23.588221 (+ 73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:23.588453 (+ 232us) raft_consensus.cc:1685] finished
0807 19:49:23.588470 (+ 17us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.588924 (+ 454us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61102,"raft.queue_time_us":47463,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":14064,"spinlock_wait_cycles":405033856,"thread_start_us":156725,"threads_started":1,"wal-append.queue_time_us":273551}
I20260807 19:49:23.982903 10944 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:23.991274 10781 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:23.992084 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:23.985736 10556 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:23.993937 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.728852 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:23.995106 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1003) took 1223 ms. Trace:
I20260807 19:49:23.995220 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.733510 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1049) took 1811 ms. Trace:
I20260807 19:49:23.975394 9015 rpcz_store.cc:276] 0807 19:49:21.928811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.929291 (+ 480us) service_pool.cc:224] Handling call
0807 19:49:23.625333 (+1696042us) spinlock_profiling.cc:243] Waited 1.08 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:23.625806 (+ 473us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.974663 (+348857us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.974937 (+ 274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2473216}
I20260807 19:49:23.750442 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38434 (request call id 179) took 5383 ms. Trace:
I20260807 19:49:23.759621 8628 catalog_manager.cc:5697] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:23.778548 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:23.782896 10877 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:23.789973 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:23.800294 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1099) took 1918 ms. Trace:
I20260807 19:49:23.802135 8790 rpcz_store.cc:276] 0807 19:49:21.396256 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.166538 (+1770282us) service_pool.cc:224] Handling call
0807 19:49:23.167018 (+ 480us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.167227 (+ 209us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.170955 (+ 3728us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:23.171013 (+ 58us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.171051 (+ 38us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.171110 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.171233 (+ 123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:23.983215 9022 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:23.997177 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:23.997929 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1103) took 1808 ms. Trace:
I20260807 19:49:23.816107 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:23.976899 9018 rpcz_store.cc:276] 0807 19:49:21.928434 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.940231 (+ 11797us) service_pool.cc:224] Handling call
0807 19:49:22.071174 (+130943us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:22.888075 (+816901us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.618086 (+730011us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b5400448754. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:23.618545 (+ 459us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876927616}
W20260807 19:49:23.999238 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 183689us
W20260807 19:49:23.999984 8849 leader_election.cc:336] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.572s (SENT)
I20260807 19:49:23.717566 10780 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:23.817874 10953 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:23.983119 8787 rpcz_store.cc:276] 0807 19:49:21.754269 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.558468 (+1804199us) service_pool.cc:224] Handling call
0807 19:49:23.559145 (+ 677us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:23.569676 (+ 10531us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.569717 (+ 41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:23.570781 (+ 1064us) log.cc:844] Serialized 26 byte log entry
0807 19:49:23.631130 (+ 60349us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.631171 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.631267 (+ 96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:23.631316 (+ 49us) raft_consensus.cc:1685] finished
0807 19:49:23.631327 (+ 11us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.631497 (+ 170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9257,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":324,"thread_start_us":69491,"threads_started":2,"wal-append.queue_time_us":46192}
I20260807 19:49:23.996543 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:23.855600 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.001695 8775 rpcz_store.cc:276] 0807 19:49:21.881815 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.804554 (+922739us) service_pool.cc:224] Handling call
0807 19:49:23.800043 (+995489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.003541 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1105) took 1807 ms. Trace:
I20260807 19:49:23.862860 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.004844 8903 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:24.005949 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.873870 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.005772 8628 catalog_manager.cc:5697] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:24.007114 8910 raft_consensus.cc:2410] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:24.000797 10780 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.008100 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.872279 8915 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:23.995934 8911 rpcz_store.cc:276] 0807 19:49:21.921507 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.030208 (+108701us) service_pool.cc:224] Handling call
0807 19:49:23.733084 (+1702876us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54001edcd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:23.733302 (+ 218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1514206080}
I20260807 19:49:23.882915 10829 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.010481 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.011360 8911 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:23.885250 10358 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:24.012784 10358 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:24.013979 10358 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:23.886678 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:23.885273 9019 rpcz_store.cc:276] 0807 19:49:21.730290 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.734174 (+ 3884us) service_pool.cc:224] Handling call
0807 19:49:23.177522 (+1443348us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:23.888067 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 714661us
I20260807 19:49:23.889644 10756 raft_consensus.cc:697] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:23.859997 8902 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:23.899655 10512 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:23.903102 8968 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:23.995460 8778 rpcz_store.cc:276] 0807 19:49:22.771785 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.842011 (+ 70226us) service_pool.cc:224] Handling call
0807 19:49:23.728445 (+886434us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:23.994999 (+266554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5300480}
I20260807 19:49:24.007613 8628 catalog_manager.cc:5697] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:23.922923 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:23.925084 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:23.925904 10695 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:23.984766 10364 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:23.946220 10860 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:23.949818 10935 raft_consensus.cc:3055] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:24.009317 10780 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:23.975498 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1101) took 1786 ms. Trace:
I20260807 19:49:23.976122 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1102) took 1786 ms. Trace:
W20260807 19:49:23.980127 8740 connection.cc:256] RPC call timeout handler was delayed by -3.2019084708299488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1816s for a response
I20260807 19:49:23.985806 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.986052 10778 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:23.994002 10781 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:23.996270 8901 raft_consensus.cc:2410] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:22.911123 8788 rpcz_store.cc:276] 0807 19:49:21.763263 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.962014 (+198751us) service_pool.cc:224] Handling call
0807 19:49:22.857431 (+895417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.053154 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1109) took 1782 ms. Trace:
I20260807 19:49:23.822273 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.054260 10881 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:24.054992 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1002) took 1283 ms. Trace:
I20260807 19:49:23.841334 9017 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:24.006821 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.049028 8972 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:24.000432 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:24.017738 8903 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:24.009327 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.019476 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:23.998293 8790 rpcz_store.cc:276] 0807 19:49:22.189873 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.997053 (+1807180us) service_pool.cc:224] Handling call
0807 19:49:23.997431 (+ 378us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.997612 (+ 181us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.997636 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:23.997675 (+ 39us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:23.997700 (+ 25us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.997751 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.997848 (+ 97us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:24.054371 8740 connection.cc:256] RPC call timeout handler was delayed by -3.173108470859006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
I20260807 19:49:23.914243 10688 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:23.918562 8913 raft_consensus.cc:1240] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.059670 9027 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.060380 10756 consensus_queue.cc:237] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.061429 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.062268 8906 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:24.063459 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.064539 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.053591 8788 rpcz_store.cc:276] 0807 19:49:22.270591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.051536 (+1780945us) service_pool.cc:224] Handling call
0807 19:49:24.052077 (+ 541us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.052622 (+ 545us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.052653 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.052714 (+ 61us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:24.052746 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.052815 (+ 69us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.052978 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.037554 10695 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:24.066545 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:24.038177 10935 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:24.066910 8853 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:24.067173 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1107) took 1868 ms. Trace:
I20260807 19:49:24.068396 10778 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:24.068136 10935 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.017866 8775 rpcz_store.cc:276] 0807 19:49:22.196168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.002536 (+1806368us) service_pool.cc:224] Handling call
0807 19:49:24.002936 (+ 400us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.003181 (+ 245us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.003212 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.003261 (+ 49us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.003290 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.003347 (+ 57us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.003475 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:23.992117 10944 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.070272 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1111) took 1766 ms. Trace:
I20260807 19:49:24.071532 10944 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.072184 10778 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:24.072615 10688 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.073684 10860 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.046593 8776 rpcz_store.cc:276] 0807 19:49:22.189299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.974279 (+1784980us) service_pool.cc:224] Handling call
0807 19:49:23.974643 (+ 364us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.974985 (+ 342us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.975016 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:23.975081 (+ 65us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:23.975120 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.975184 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.975314 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.071985 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:24.070715 8788 rpcz_store.cc:276] 0807 19:49:22.303818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.065637 (+1761819us) service_pool.cc:224] Handling call
0807 19:49:24.066151 (+ 514us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.069772 (+ 3621us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.069805 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.069932 (+ 127us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:24.069972 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.070052 (+ 80us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.070182 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:23.921430 10705 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:23.996281 8630 rpcz_store.cc:276] 0807 19:49:18.366672 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:18.366950 (+ 278us) service_pool.cc:224] Handling call
0807 19:49:18.412537 (+ 45587us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b540011a8d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56e470 00007f687b60d31a 00007f68813fd38c 00007f68814f9bfe 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc 00007f687f9b2592 00007f687993e07c 00007f687993d4b6
0807 19:49:23.750159 (+5337622us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:21.594965 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:21.699838 (+104873us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:21.699876 (+ 38us) write_op.cc:435] Acquired schema lock
0807 19:49:21.699894 (+ 18us) tablet.cc:662] Decoding operations
0807 19:49:21.700222 (+ 328us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:21.700307 (+ 85us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:21.700358 (+ 51us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:49:21.700563 (+ 205us) tablet.cc:701] Row locks acquired
0807 19:49:21.700581 (+ 18us) write_op.cc:260] PREPARE: finished
0807 19:49:21.700722 (+ 141us) write_op.cc:270] Start()
0807 19:49:21.700827 (+ 105us) write_op.cc:276] Timestamp: P: 1786132161700678 usec, L: 0
0807 19:49:21.700845 (+ 18us) op_driver.cc:348] REPLICATION: starting
0807 19:49:21.701607 (+ 762us) log.cc:844] Serialized 532 byte log entry
0807 19:49:21.782289 (+ 80682us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f68784e0a5c 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb
0807 19:49:22.882432 (+1100143us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b200001dbf8. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b1b1d71 00007f687b1d3bc9 00007f687b19eab3 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359 00007f687b1a7321 00007f687b1a661d 00000000003aa554
0807 19:49:23.206416 (+323984us) op_driver.cc:464] REPLICATION: finished
0807 19:49:23.224551 (+ 18135us) write_op.cc:301] APPLY: starting
0807 19:49:23.261760 (+ 37209us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:23.262088 (+ 328us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:23.262147 (+ 59us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:23.262599 (+ 452us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:23.263102 (+ 503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:49:23.263181 (+ 79us) write_op.cc:312] APPLY: finished
0807 19:49:23.263731 (+ 550us) log.cc:844] Serialized 22 byte log entry
0807 19:49:23.264134 (+ 403us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:23.264381 (+ 247us) write_op.cc:454] Released schema lock
0807 19:49:23.264443 (+ 62us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:23.265364 (+ 921us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:23.265406 (+ 42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":80346880,"child_traces":[["op",{"apply.queue_time_us":17797,"apply.run_cpu_time_us":3911,"apply.run_wall_time_us":40700,"num_ops":1,"prepare.queue_time_us":222462,"prepare.run_cpu_time_us":49124,"prepare.run_wall_time_us":283924,"raft.queue_time_us":306501,"raft.run_cpu_time_us":8198,"raft.run_wall_time_us":69444,"replication_time_us":1501506,"spinlock_wait_cycles":47976576,"thread_start_us":518646,"threads_started":4,"wal-append.queue_time_us":219077}]]}
W20260807 19:49:24.028520 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:24.078979 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.079340 8788 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.032941 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 790) took 1124 ms. Trace:
I20260807 19:49:24.077468 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.080576 10626 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:23.955757 10782 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.081166 10626 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:24.081600 10626 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.082911 10782 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.001086 10953 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:24.035979 10512 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.084475 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 45) took 1311 ms. Trace:
W20260807 19:49:24.037526 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 134576us
I20260807 19:49:24.055341 8780 rpcz_store.cc:276] 0807 19:49:22.771334 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.840916 (+ 69582us) service_pool.cc:224] Handling call
0807 19:49:23.821927 (+981011us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:24.054912 (+232985us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18561920}
I20260807 19:49:23.975739 8904 rpcz_store.cc:276] 0807 19:49:21.885462 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.887794 (+ 2332us) service_pool.cc:224] Handling call
0807 19:49:21.951954 (+ 64160us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:22.860562 (+908608us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:22.865236 (+ 4674us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:22.991587 (+126351us) log.cc:844] Serialized 26 byte log entry
0807 19:49:23.622257 (+630670us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.622386 (+ 129us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.622551 (+ 165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:23.622630 (+ 79us) raft_consensus.cc:1685] finished
0807 19:49:23.622654 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.622885 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":257258,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":144,"thread_start_us":1222287,"threads_started":2,"wal-append.queue_time_us":125669}
I20260807 19:49:23.982105 10950 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.046703 8783 rpcz_store.cc:276] 0807 19:49:22.189620 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.975035 (+1785415us) service_pool.cc:224] Handling call
0807 19:49:23.975551 (+ 516us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:23.975841 (+ 290us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.975866 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:23.975904 (+ 38us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:23.975929 (+ 25us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.975980 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.976071 (+ 91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.095551 8968 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:24.096956 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 48) took 1226 ms. Trace:
I20260807 19:49:24.049731 10364 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:24.050645 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.097867 10364 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.098531 9893 raft_consensus.cc:1240] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:24.098922 10782 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 19:49:23.454269 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.099076 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:24.099975 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 52594.1us
I20260807 19:49:24.100977 10392 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:24.102025 10741 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:24.102561 10741 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:24.053629 8856 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:24.057703 8740 connection.cc:256] RPC call timeout handler was delayed by -2.8086084709502757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22530000000000003s for a response
I20260807 19:49:24.101189 10364 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.055307 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:24.028365 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:24.071991 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 72096.7us
I20260807 19:49:23.973634 8908 rpcz_store.cc:276] 0807 19:49:21.901086 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.144155 (+243069us) service_pool.cc:224] Handling call
0807 19:49:22.144565 (+ 410us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:22.868926 (+724361us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:22.868956 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:22.869412 (+ 456us) log.cc:844] Serialized 26 byte log entry
0807 19:49:23.020376 (+150964us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.020422 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.020513 (+ 91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:23.616079 (+595566us) raft_consensus.cc:1685] finished
0807 19:49:23.616120 (+ 41us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:23.616479 (+ 359us) inbound_call.cc:177] Queueing success response
0807 19:49:23.795723 (+179244us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b48000c03d8. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6880560211 00007f68805572a7 00007f688055713a 00007f688064e07f 00007f688064dd47 00007f688066236c 00007f6880662318 00007f6880662298 00007f6880662253 00007f68806614fc 00007f687f589176
Metrics: {"raft.queue_time_us":252116,"raft.run_cpu_time_us":15062,"raft.run_wall_time_us":1386566,"spinlock_wait_cycles":7913856,"thread_start_us":842918,"threads_started":2,"wal-append.queue_time_us":298049}
I20260807 19:49:24.077381 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1110) took 1774 ms. Trace:
I20260807 19:49:24.085294 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.027835 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.067651 8778 rpcz_store.cc:276] 0807 19:49:22.199121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.029039 (+1829918us) service_pool.cc:224] Handling call
0807 19:49:24.066540 (+ 37501us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.066829 (+ 289us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.066853 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.066894 (+ 41us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:24.066925 (+ 31us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.066981 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.067087 (+ 106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776576}
I20260807 19:49:24.080678 8902 rpcz_store.cc:276] 0807 19:49:22.900919 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.172554 (+271635us) service_pool.cc:224] Handling call
0807 19:49:24.021630 (+849076us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:24.092614 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.110482 8790 rpcz_store.cc:276] 0807 19:49:22.303481 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.076417 (+1772936us) service_pool.cc:224] Handling call
0807 19:49:24.076813 (+ 396us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.077058 (+ 245us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.077082 (+ 24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.077128 (+ 46us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:24.077153 (+ 25us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.077204 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.077299 (+ 95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.087294 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 85) took 1272 ms. Trace:
W20260807 19:49:24.087220 10954 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.073s user 0.002s sys 0.000s
I20260807 19:49:24.093845 8911 raft_consensus.cc:2393] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:24.095028 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 86) took 1280 ms. Trace:
I20260807 19:49:24.063453 8628 catalog_manager.cc:5697] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:24.097826 8972 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:24.097386 8783 rpcz_store.cc:276] 0807 19:49:22.870246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.095803 (+1225557us) service_pool.cc:224] Handling call
0807 19:49:24.096264 (+ 461us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.096599 (+ 335us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.096621 (+ 22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.096666 (+ 45us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.096695 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.096759 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.096897 (+ 138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.099274 8901 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.112692 8906 rpcz_store.cc:276] 0807 19:49:22.814442 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.086114 (+1271672us) service_pool.cc:224] Handling call
0807 19:49:24.086583 (+ 469us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.086916 (+ 333us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.086942 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.087001 (+ 59us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.087030 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.087090 (+ 60us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.087212 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:24.110580 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 57168.9us
I20260807 19:49:24.121439 8903 raft_consensus.cc:1240] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260807 19:49:24.070771 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1112) took 1766 ms. Trace:
I20260807 19:49:24.123646 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 783) took 1229 ms. Trace:
I20260807 19:49:24.124076 8901 rpcz_store.cc:276] 0807 19:49:22.886002 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.098669 (+1212667us) service_pool.cc:224] Handling call
0807 19:49:24.099055 (+ 386us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.114854 (+ 15799us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.115052 (+ 198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.126286 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 791) took 1213 ms. Trace:
I20260807 19:49:24.093135 10944 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:24.126614 8903 rpcz_store.cc:276] 0807 19:49:22.913212 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.107120 (+1193908us) service_pool.cc:224] Handling call
0807 19:49:24.107618 (+ 498us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:24.125999 (+ 18381us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.126235 (+ 236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.111842 10950 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:24.085104 8788 rpcz_store.cc:276] 0807 19:49:22.772691 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.078589 (+1305898us) service_pool.cc:224] Handling call
0807 19:49:24.079047 (+ 458us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.079852 (+ 805us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.080097 (+ 245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.132702 10364 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:24.139420 8915 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:24.139456 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 84) took 1325 ms. Trace:
I20260807 19:49:24.127280 8775 rpcz_store.cc:276] 0807 19:49:22.304103 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.069711 (+1765608us) service_pool.cc:224] Handling call
0807 19:49:24.070156 (+ 445us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.070412 (+ 256us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.070443 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.070484 (+ 41us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:24.070541 (+ 57us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.070598 (+ 57us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.070712 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.146984 8902 raft_consensus.cc:1240] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:24.139329 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.123031 10705 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:24.135970 8972 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:24.141634 8913 rpcz_store.cc:276] 0807 19:49:22.814143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.882489 (+ 68346us) service_pool.cc:224] Handling call
0807 19:49:22.882786 (+ 297us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.096287 (+1213501us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.129215 (+ 32928us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54003610d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:24.139391 (+ 10176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381117696}
I20260807 19:49:24.158583 9027 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:24.135423 10935 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:24.157001 8849 leader_election.cc:336] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.582s (SENT)
I20260807 19:49:24.173095 9023 raft_consensus.cc:1240] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:24.178292 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:24.178058 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 77438.1us
I20260807 19:49:24.180606 8913 raft_consensus.cc:1217] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2] Dedup: 2.2->[]
I20260807 19:49:24.180837 8906 raft_consensus.cc:1240] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:24.103674 10741 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.158967 8628 catalog_manager.cc:5697] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:24.181280 8913 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:24.176821 10364 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:24.185992 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 50132.3us
W20260807 19:49:24.186645 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 79102.5us
I20260807 19:49:24.173632 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 44) took 1392 ms. Trace:
I20260807 19:49:24.191947 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 49) took 1312 ms. Trace:
I20260807 19:49:24.194635 8780 rpcz_store.cc:276] 0807 19:49:22.879160 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.108286 (+1229126us) service_pool.cc:224] Handling call
0807 19:49:24.156555 (+ 48269us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.191613 (+ 35058us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.191639 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.191678 (+ 39us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.191706 (+ 28us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.191770 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.191882 (+ 112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.198160 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1113) took 1291 ms. Trace:
I20260807 19:49:24.198561 8783 rpcz_store.cc:276] 0807 19:49:22.906805 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.128984 (+1222179us) service_pool.cc:224] Handling call
0807 19:49:24.182130 (+ 53146us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.197806 (+ 15676us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.197837 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.197896 (+ 59us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:24.197929 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.197990 (+ 61us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.198101 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.200173 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 50) took 1320 ms. Trace:
I20260807 19:49:24.199664 8915 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:24.190171 10787 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
W20260807 19:49:24.203495 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 63008.5us
I20260807 19:49:24.200609 8790 rpcz_store.cc:276] 0807 19:49:22.879497 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.123371 (+1243874us) service_pool.cc:224] Handling call
0807 19:49:24.156271 (+ 32900us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.187752 (+ 31481us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.187782 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.191528 (+ 3746us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.199935 (+ 8407us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.200002 (+ 67us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.200115 (+ 113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.113837 8904 rpcz_store.cc:276] 0807 19:49:22.814696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.093791 (+1279095us) service_pool.cc:224] Handling call
0807 19:49:24.094233 (+ 442us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.094561 (+ 328us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.094591 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.094671 (+ 80us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.094721 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.094789 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.094958 (+ 169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.165454 10950 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.165793 10781 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260807 19:49:24.179214 10944 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.209762 8911 raft_consensus.cc:1240] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:24.178082 10953 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260807 19:49:24.213434 8910 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.203397 10778 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:24.214658 10364 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.218042 10626 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:24.218816 9838 leader_election.cc:336] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.029s (SENT)
I20260807 19:49:24.219627 10915 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:24.220295 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 80298.7us
I20260807 19:49:24.244946 8968 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:24.246150 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1088) took 2445 ms. Trace:
I20260807 19:49:24.250222 8915 raft_consensus.cc:1275] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:24.249776 8789 rpcz_store.cc:276] 0807 19:49:21.767998 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.818393 (+2050395us) service_pool.cc:224] Handling call
0807 19:49:23.818901 (+ 508us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:23.819452 (+ 551us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:23.819478 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:23.820054 (+ 576us) log.cc:844] Serialized 26 byte log entry
0807 19:49:23.961612 (+141558us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:23.961663 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:23.961749 (+ 86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:24.086447 (+124698us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:24.209631 (+123184us) raft_consensus.cc:1685] finished
0807 19:49:24.209659 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.212846 (+ 3187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"spinlock_wait_cycles":10153728,"thread_start_us":140942,"threads_started":1,"wal-append.queue_time_us":193951}
I20260807 19:49:24.272028 8911 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:24.272081 8902 raft_consensus.cc:1240] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 3. Current term is 4. Ops: []
W20260807 19:49:24.273882 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60755.2us
W20260807 19:49:24.279006 8849 leader_election.cc:336] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.691s (SENT)
W20260807 19:49:24.279335 8970 consensus_peers.cc:597] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 -> Peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143): Couldn't send request to peer 39975243456d4452b4a4870ac7ebafc5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a898110875ba40ab8e512e124aff4fb2. This is attempt 1: this message will repeat every 5th retry.
W20260807 19:49:24.279886 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 55109.8us
I20260807 19:49:24.253106 10915 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:24.289788 8856 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
W20260807 19:49:24.304922 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 109633us
I20260807 19:49:24.193861 8776 rpcz_store.cc:276] 0807 19:49:22.772432 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.075265 (+1302833us) service_pool.cc:224] Handling call
0807 19:49:24.075693 (+ 428us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:24.114225 (+ 38532us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.122360 (+ 8135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:24.131237 (+ 8877us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:24.136189 (+ 4952us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.140901 (+ 4712us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.165083 (+ 24182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:24.245913 10860 consensus_queue.cc:1059] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260807 19:49:24.318440 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:24.314982 8903 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:24.327286 10778 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:24.329232 10695 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:24.332135 10695 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.334293 10695 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.330188 9023 raft_consensus.cc:1275] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:24.330864 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 86150.8us
I20260807 19:49:24.337577 8968 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:24.340986 10695 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:24.341482 10695 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:24.341930 10695 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:24.326702 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:24.345399 10963 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:24.345837 10963 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:24.346186 10963 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:24.347764 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 72182.2us
I20260807 19:49:24.352444 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:24.359687 10780 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:24.360392 10950 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:24.361408 10765 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:24.361897 10765 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:24.362293 10765 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:24.362658 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:24.363315 10782 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:24.293571 10915 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:24.361465 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 82509.5us
I20260807 19:49:24.367760 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:24.357576 10778 raft_consensus.cc:697] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:24.393636 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 79212.9us
I20260807 19:49:24.366948 10782 raft_consensus.cc:697] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:24.424065 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.366513 10787 consensus_queue.cc:1059] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260807 19:49:24.432971 8968 leader_election.cc:400] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3.
I20260807 19:49:24.397117 10950 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:24.440825 8849 leader_election.cc:336] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.698s (SENT)
I20260807 19:49:24.396840 10974 consensus_queue.cc:1059] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260807 19:49:24.434110 10556 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:24.448578 10778 consensus_queue.cc:237] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:24.449409 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 60421.4us
I20260807 19:49:24.448863 8628 catalog_manager.cc:5697] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
W20260807 19:49:24.450083 10972 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.157s user 0.000s sys 0.001s
I20260807 19:49:24.454429 10988 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:24.455103 10626 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:24.456073 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.456559 9023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:24.448271 10978 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:24.456097 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 104870us
I20260807 19:49:24.466089 9023 raft_consensus.cc:606] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
W20260807 19:49:24.467446 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 70255.5us
W20260807 19:49:24.468159 8849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.056s (SENT) [suppressed 62 similar messages]
I20260807 19:49:24.468935 10974 consensus_queue.cc:1243] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer 39975243456d4452b4a4870ac7ebafc5 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260807 19:49:24.437039 10688 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.470014 10782 consensus_queue.cc:237] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.471628 9018 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.472296 10364 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:24.472302 10988 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.473672 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:24.474519 10963 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.475025 10988 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.475374 10358 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:24.475776 10358 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:24.475916 10626 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:24.476647 10556 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.476979 10765 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.476187 10358 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:24.478790 10556 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.478937 10765 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.457201 10895 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:24.479610 10895 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.482167 10895 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.476468 10950 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.466536 9023 raft_consensus.cc:614] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:24.446300 10944 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:24.494422 9062 debug-util.cc:398] Leaking SignalData structure 0x7b080009a100 after lost signal to thread 6315
W20260807 19:49:24.494875 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08000a9ae0 after lost signal to thread 6318
W20260807 19:49:24.495882 9062 debug-util.cc:398] Leaking SignalData structure 0x7b080004ad40 after lost signal to thread 8695
I20260807 19:49:24.496636 10974 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:24.496862 10963 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.497313 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.497342 10974 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:24.462805 10741 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260807 19:49:24.499891 10973 raft_consensus.cc:3055] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:24.500377 10973 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:24.485080 10950 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.510831 9893 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 19:49:24.512859 8628 catalog_manager.cc:5697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:24.520256 9023 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 3. Ops: []
I20260807 19:49:24.525141 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.507799 10895 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:24.515904 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2540 after lost signal to thread 8828
I20260807 19:49:24.529249 9022 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 3. Ops: []
I20260807 19:49:24.519577 9021 raft_consensus.cc:1240] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:24.515748 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
W20260807 19:49:24.532409 10968 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.250s user 0.000s sys 0.001s
I20260807 19:49:24.499900 10974 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:24.547173 10980 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
I20260807 19:49:24.545011 10829 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260807 19:49:24.501457 10973 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.521282 9008 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:24.538506 10977 raft_consensus.cc:3055] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:24.549141 10977 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:24.544643 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2500 after lost signal to thread 8831
I20260807 19:49:24.518960 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.515129 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1100) took 2633 ms. Trace:
I20260807 19:49:24.550241 10977 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.551594 10977 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.552800 10895 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:24.553481 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08003a0000 after lost signal to thread 8951
W20260807 19:49:24.553849 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08003443c0 after lost signal to thread 8952
W20260807 19:49:24.555094 9062 debug-util.cc:398] Leaking SignalData structure 0x7b0800283000 after lost signal to thread 9068
I20260807 19:49:24.544188 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:24.515676 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 388088us
I20260807 19:49:24.556985 10950 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:24.547092 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1104) took 2357 ms. Trace:
I20260807 19:49:24.561185 10705 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:24.562058 9017 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:24.561299 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.561839 8787 rpcz_store.cc:276] 0807 19:49:22.195811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.001765 (+1805954us) service_pool.cc:224] Handling call
0807 19:49:24.002230 (+ 465us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:24.002722 (+ 492us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.002750 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:24.003373 (+ 623us) log.cc:844] Serialized 26 byte log entry
0807 19:49:24.118541 (+115168us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:24.118588 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.118672 (+ 84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:24.531572 (+412900us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:24.546798 (+ 15226us) raft_consensus.cc:1685] finished
0807 19:49:24.546821 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.547031 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19358,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":21581,"spinlock_wait_cycles":2790784,"thread_start_us":114739,"threads_started":1,"wal-append.queue_time_us":189920}
I20260807 19:49:24.574931 9018 raft_consensus.cc:1275] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:24.542590 10688 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:24.518285 10976 log.cc:927] Time spent T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.225s user 0.005s sys 0.003s
I20260807 19:49:24.550920 8779 rpcz_store.cc:276] 0807 19:49:21.882127 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.971941 (+2089814us) service_pool.cc:224] Handling call
0807 19:49:23.972324 (+ 383us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:24.110279 (+137955us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.110314 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:24.136927 (+ 26613us) log.cc:844] Serialized 26 byte log entry
0807 19:49:24.355500 (+218573us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:24.355549 (+ 49us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.355621 (+ 72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:24.514900 (+159279us) raft_consensus.cc:1685] finished
0807 19:49:24.514920 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.515044 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6923,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":176,"thread_start_us":247763,"threads_started":2,"wal-append.queue_time_us":321374}
W20260807 19:49:24.601164 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.544210 8628 catalog_manager.cc:5697] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
W20260807 19:49:24.558940 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08003934a0 after lost signal to thread 9937
I20260807 19:49:24.607113 10950 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.562037 10695 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.602066 10781 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:24.585994 10915 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.613508 8908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 794) took 1425 ms. Trace:
I20260807 19:49:24.618148 8908 rpcz_store.cc:276] 0807 19:49:23.188026 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.113739 (+925713us) service_pool.cc:224] Handling call
0807 19:49:24.131196 (+ 17457us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:24.146017 (+ 14821us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:24.146045 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:24.146635 (+ 590us) log.cc:844] Serialized 26 byte log entry
0807 19:49:24.177542 (+ 30907us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:24.177582 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:24.177663 (+ 81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:24.517495 (+339832us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:24.613157 (+ 95662us) raft_consensus.cc:1685] finished
0807 19:49:24.613176 (+ 19us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:24.613401 (+ 225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24812800,"thread_start_us":30437,"threads_started":1,"wal-append.queue_time_us":50805}
I20260807 19:49:24.618889 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.575698 10895 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.621405 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.583101 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.626608 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:24.629765 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 206424us
I20260807 19:49:24.610165 10973 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:24.610021 9062 debug-util.cc:398] Leaking SignalData structure 0x7b08003934c0 after lost signal to thread 9938
W20260807 19:49:24.634794 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 166655us
W20260807 19:49:24.641436 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.615021 10915 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:24.648981 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 171219us
I20260807 19:49:24.612962 10695 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.652431 10953 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:24.648629 9008 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:24.652213 10895 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260807 19:49:24.611095 10358 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.665297 9020 raft_consensus.cc:1275] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:24.636986 9062 debug-util.cc:398] Leaking SignalData structure 0x7b080031f700 after lost signal to thread 10384
I20260807 19:49:24.613518 10781 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.613909 8628 catalog_manager.cc:5697] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
W20260807 19:49:24.664796 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 155446us
W20260807 19:49:24.672195 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.654568 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:24.677326 8849 leader_election.cc:336] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.742s (SENT)
I20260807 19:49:24.680754 9017 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 19:49:24.685840 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.666745 9022 raft_consensus.cc:1275] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:24.661921 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 104726us
I20260807 19:49:24.693727 8849 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:24.692006 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.696012 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.696408 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.699003 8902 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:24.696056 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.678715 10780 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:24.701684 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:24.696233 9019 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 3. Ops: []
I20260807 19:49:24.705919 8628 catalog_manager.cc:5697] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:24.706609 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.708388 8910 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:24.709362 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.709628 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.710006 8903 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:24.710192 8901 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:24.710630 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:24.709395 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.711596 8913 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.708930 10894 consensus_queue.cc:1059] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260807 19:49:24.713029 10819 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:24.707541 10705 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:24.715132 10705 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:24.715566 10705 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:24.715899 10384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba47e79caa54c409c53304a66ce0160. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 19:49:24.700001 10992 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.210s user 0.003s sys 0.000s
W20260807 19:49:24.720965 10982 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.252s user 0.003s sys 0.000s
I20260807 19:49:24.722147 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.722820 10935 raft_consensus.cc:3055] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:24.723201 10935 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:24.724161 10935 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.724730 9027 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.727407 10358 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:24.727559 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.729768 10950 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.697304 10741 consensus_queue.cc:1059] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260807 19:49:24.712641 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:24.731194 8906 raft_consensus.cc:2393] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:24.734004 8911 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:24.735296 8915 raft_consensus.cc:2410] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260807 19:49:24.732394 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:24.740397 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.744262 10895 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260807 19:49:24.744388 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 73345.3us
W20260807 19:49:24.748441 8849 leader_election.cc:336] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.811s (SENT)
I20260807 19:49:24.729892 10781 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.733693 10778 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:24.753106 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.763319 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:24.766008 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51501.1us
I20260807 19:49:24.766990 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.767869 9017 raft_consensus.cc:2393] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:24.773104 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.783718 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.784076 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.784828 9008 raft_consensus.cc:2393] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:24.785458 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.785826 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.784693 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.786171 9021 raft_consensus.cc:2393] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:24.787070 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.787147 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.788278 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.788162 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.788694 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.789304 9021 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:24.793062 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:24.796693 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.799593 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.799396 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:24.800854 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.801597 9022 raft_consensus.cc:2410] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260807 19:49:24.802175 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.802591 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.802863 9022 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.803136 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.803362 8776 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:24.803768 8787 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 1.
I20260807 19:49:24.804733 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.806197 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.802214 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:24.807629 8911 raft_consensus.cc:1240] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.807433 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.808214 9023 raft_consensus.cc:2393] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:24.811532 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.757349 10950 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260807 19:49:24.814100 9021 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.804291 9022 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.835763 10973 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:24.836378 10973 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:24.836824 10973 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:24.833340 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.838408 8790 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.835927 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.842990 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.845057 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.845815 8780 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.846045 8740 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:24.846923 10977 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:24.847335 10977 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:24.847479 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:24.847767 10977 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:24.861857 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:24.836519 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.863325 11000 raft_consensus.cc:3055] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:24.867395 10950 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:24.867954 8788 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:24.869016 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.870415 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.865324 11000 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:24.872174 10973 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.872663 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.873217 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.873956 8787 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:24.873380 11000 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:24.874763 10973 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:24.875244 11000 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:24.876284 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:24.872566 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.877779 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.882401 8783 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:24.877084 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:24.885720 8775 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:24.865752 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:24.931121 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.933051 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:24.935891 9020 raft_consensus.cc:1240] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.937119 9019 raft_consensus.cc:1240] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:24.938776 9020 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:24.940476 10778 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:24.940912 10778 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:24.941421 8849 leader_election.cc:336] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items
I20260807 19:49:24.941896 10778 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:24.943284 10778 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:24.941946 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:24.958292 10384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:33436 (request call id 103) took 8766 ms. Trace:
I20260807 19:49:24.963461 10950 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:24.965413 9017 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:24.970937 8779 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:24.983060 8780 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:24.973742 8783 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:24.971910 8790 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:24.990070 10974 raft_consensus.cc:2764] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260807 19:49:24.990602 9017 raft_consensus.cc:1240] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:24.992028 9017 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
W20260807 19:49:24.995066 8849 leader_election.cc:336] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51022 dropped due to backpressure: service queue is full with 50 items
I20260807 19:49:24.978976 10384 rpcz_store.cc:276] 0807 19:49:16.191677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.195119 (+ 3442us) service_pool.cc:224] Handling call
0807 19:49:16.231885 (+ 36766us) ts_tablet_manager.cc:934] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
0807 19:49:24.714208 (+8482323us) tablet_copy_client.cc:368] Tablet copy session begun
0807 19:49:24.819096 (+104888us) tablet_metadata.cc:665] Metadata flushed
0807 19:49:24.819243 (+ 147us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 19:49:24.957210 (+137967us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 19:49:24.957592 (+ 382us) tablet_replica.cc:206] Creating consensus instance
0807 19:49:24.958020 (+ 428us) ts_tablet_manager.cc:989] Replica registered.
0807 19:49:24.958173 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":61985,"tablet-copy.queue_time_us":9313,"thread_start_us":625082,"threads_started":1}
I20260807 19:49:24.995601 8849 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:25.017277 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 168228us
I20260807 19:49:24.982044 10973 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.022794 9017 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:24.996426 10384 tablet_copy_client.cc:806] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:25.025710 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:25.027722 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:25.029071 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.012185 10950 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:25.029711 8778 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:25.032181 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.033135 10950 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:25.034301 8849 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.037343 8775 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:25.038640 10780 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.039161 10780 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.039585 10780 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:25.041632 10950 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:25.042366 10384 tablet_copy_client.cc:670] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:25.020278 10973 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:25.054589 8775 raft_consensus.cc:3055] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:25.056793 10973 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:25.072290 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 140890us
I20260807 19:49:25.077540 9017 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260807 19:49:24.885429 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 10901e14a5e8471291ad1d7c203b6740 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:25.057155 8775 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:25.044519 10705 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:25.086505 8778 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 19:49:25.090458 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:25.092058 9027 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:25.097663 8788 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:25.098498 8911 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
W20260807 19:49:25.100740 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.101361 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.108703 11019 ts_tablet_manager.cc:933] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.115824 10977 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.117136 8789 raft_consensus.cc:1240] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:25.117409 8849 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:25.118655 8775 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.124317 8775 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:25.124451 10780 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.136876 10780 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.138963 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:25.140415 9020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:25.141028 9020 raft_consensus.cc:606] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:25.143323 8790 raft_consensus.cc:1240] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:25.122174 10781 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:25.122357 10978 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.149502 8788 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:25.149330 10978 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.148309 10977 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.183892 8789 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:25.177371 10978 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:25.187556 11019 tablet_copy_client.cc:323] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.42.195:35801
I20260807 19:49:25.191489 8849 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.194777 9022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:25.180862 10935 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:25.200052 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:25.202953 11029 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.078s user 0.000s sys 0.001s
I20260807 19:49:25.203495 10895 consensus_queue.cc:1059] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 19:49:25.206014 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.202852 8968 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.209393 8968 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.210075 10780 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:25.210536 10780 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.210902 10780 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:25.214083 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:25.217381 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.217945 8968 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.213692 10705 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:25.216488 10364 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:25.222506 10688 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:25.223121 10688 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:25.224076 10688 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.228363 8968 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.207036 9022 raft_consensus.cc:606] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:25.220682 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:25.215637 10915 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:25.233353 8968 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.222427 10364 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:25.238740 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 47355.6us
I20260807 19:49:25.240152 10358 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:25.236776 10705 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260807 19:49:25.237270 10950 raft_consensus.cc:3055] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:25.240986 10950 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:25.241463 8849 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.216148 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:25.242920 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.211596 10985 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260807 19:49:25.228214 10829 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:25.230942 9022 raft_consensus.cc:614] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:25.250144 8968 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.240749 10988 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:25.237187 10556 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.238157 10915 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:25.256790 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:25.248263 10364 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:25.241869 10950 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.260798 10765 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260807 19:49:25.268232 10695 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:25.253423 8775 raft_consensus.cc:1275] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:25.253702 10974 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.274698 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:25.251816 9019 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:25.254931 10782 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:25.252077 10358 raft_consensus.cc:697] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:25.289212 10358 consensus_queue.cc:237] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.260789 10556 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.260818 10953 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260807 19:49:25.292755 10556 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:25.293747 10950 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:25.265291 10915 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:25.296443 11000 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:49:25.297328 11000 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:25.256117 10692 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260807 19:49:25.297652 10974 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.298285 10974 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:25.292781 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51464.3us
W20260807 19:49:25.299945 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.301095 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
W20260807 19:49:25.302093 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.292035 10780 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:25.314239 10780 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:25.316891 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 51856.1us
I20260807 19:49:25.297518 10765 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 19:49:25.323701 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.330160 10705 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:25.297676 11000 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:25.307761 10977 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:25.318852 11053 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.086s user 0.001s sys 0.000s
W20260807 19:49:25.336629 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:25.348557 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet be89699a1aff43f1a97b936b3154c2d5 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:25.349433 10819 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:25.349838 10819 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:25.350020 9893 raft_consensus.cc:1275] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 19:49:25.350989 10978 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.316509 10392 consensus_queue.cc:1059] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260807 19:49:25.353242 10978 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.357124 10996 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:25.344456 10780 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.358316 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.358670 10780 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:25.360067 10780 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:25.418501 9021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:25.419237 9021 raft_consensus.cc:606] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:25.342764 10692 consensus_queue.cc:1059] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
W20260807 19:49:25.435956 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 117089us
I20260807 19:49:25.363830 10978 raft_consensus.cc:2764] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260807 19:49:25.347815 10915 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.390363 10782 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:25.442106 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 204761us
W20260807 19:49:25.453317 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.414634 10781 consensus_queue.cc:1048] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260807 19:49:25.498509 9023 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:25.441367 10988 raft_consensus.cc:697] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:25.508512 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:25.418701 11070 consensus_queue.cc:1059] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260807 19:49:25.464252 10765 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:25.500672 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 140493us
W20260807 19:49:25.513212 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:25.517482 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 216500us
W20260807 19:49:25.513262 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 76336.7us
I20260807 19:49:25.436777 11000 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:25.511114 10358 raft_consensus.cc:3055] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
W20260807 19:49:25.510133 9838 leader_election.cc:336] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.504s (SENT)
I20260807 19:49:25.471441 10978 consensus_queue.cc:1059] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
W20260807 19:49:25.542814 11058 log.cc:927] Time spent T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.096s user 0.001s sys 0.000s
I20260807 19:49:25.515324 9044 tablet_copy_service.cc:140] P a38f4d6088cf4c8cb510200ca0555801: Received BeginTabletCopySession request for tablet a898110875ba40ab8e512e124aff4fb2 from peer 39975243456d4452b4a4870ac7ebafc5 ({username='slave'} at 127.0.0.1:37380)
I20260807 19:49:25.543695 10765 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.501684 10819 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.545677 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.546281 10556 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.526957 10358 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:25.520104 11070 consensus_queue.cc:1243] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer 671d8754170646eaac5748745eba58b4 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 19:49:25.500317 11060 consensus_queue.cc:1059] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
W20260807 19:49:25.559577 11071 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.086s user 0.001s sys 0.000s
I20260807 19:49:25.560920 10819 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:25.562897 9044 tablet_copy_service.cc:161] P a38f4d6088cf4c8cb510200ca0555801: Beginning new tablet copy session on tablet a898110875ba40ab8e512e124aff4fb2 from peer 39975243456d4452b4a4870ac7ebafc5 at {username='slave'} at 127.0.0.1:37380: session id = 39975243456d4452b4a4870ac7ebafc5-a898110875ba40ab8e512e124aff4fb2
I20260807 19:49:25.563606 10988 consensus_queue.cc:237] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.565083 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:25.569005 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.555855 11000 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:25.568240 10973 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.502s user 0.012s sys 0.000s
I20260807 19:49:25.569397 10781 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:25.602284 10781 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.602721 10781 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:25.603690 10973 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:25.604878 10915 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:25.584570 11070 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260807 19:49:25.606253 11070 consensus_queue.cc:1048] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:49:25.588949 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 78871.8us
I20260807 19:49:25.609980 10935 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:25.610522 10935 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:25.591267 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.613794 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:25.606348 10973 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:25.614717 9008 raft_consensus.cc:606] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:25.615666 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:25.595546 11083 consensus_queue.cc:1059] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260807 19:49:25.596674 10358 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.555675 10782 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.601680 9019 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:25.618901 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 19:49:25.631665 10556 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.632180 11065 raft_consensus.cc:3055] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
W20260807 19:49:25.633452 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 68553.7us
I20260807 19:49:25.634584 11060 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:25.636570 11083 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
W20260807 19:49:25.648459 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 118648us
I20260807 19:49:25.675516 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:25.670485 11083 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:25.678162 11083 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.679448 11083 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:25.681710 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.683180 10780 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:25.684321 8908 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:25.669457 11060 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:25.685122 11060 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:25.685647 10935 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.686844 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.667464 11065 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:25.670650 10358 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:25.688334 8849 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.684962 9018 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260807 19:49:25.688486 11064 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 19:49:25.708148 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.724797 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:25.725183 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.725597 8908 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
W20260807 19:49:25.731098 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.690160 10556 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:25.679991 10364 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.707098 10895 raft_consensus.cc:3055] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:25.728125 11055 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:25.730536 10950 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:25.698357 11065 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:25.699093 10935 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260807 19:49:25.731010 10780 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:25.744978 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:25.733217 10556 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:25.756738 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 235715us
W20260807 19:49:25.761906 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 80281.7us
I20260807 19:49:25.762761 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.764158 9850 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:25.765872 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 77670.1us
I20260807 19:49:25.738124 10895 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:25.772086 8788 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:25.742558 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:25.778373 9019 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:25.753775 11082 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:25.775501 11000 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:25.750470 10974 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.800824 11065 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:25.811031 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.808990 11076 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:25.812575 11076 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.814304 8779 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:25.814862 11076 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:25.815670 8779 raft_consensus.cc:1240] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:25.801330 10977 raft_consensus.cc:3055] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:25.822156 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:25.822875 8778 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:25.807356 11000 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:25.831784 8787 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:25.757855 11055 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:25.820837 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.754213 11090 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260807 19:49:25.845752 11072 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260807 19:49:25.819931 10977 raft_consensus.cc:740] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:25.831048 10978 consensus_queue.cc:1243] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer 671d8754170646eaac5748745eba58b4 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 19:49:25.864393 8780 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:25.839051 8776 pending_rounds.cc:85] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:25.838421 11000 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:25.831017 11081 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:25.804124 11082 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.869241 8776 pending_rounds.cc:107] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260807 19:49:25.937322 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 174663us
I20260807 19:49:25.938920 8740 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.824113 10895 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:25.952666 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.841485 10556 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.937196 8776 raft_consensus.cc:2889] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:25.936422 10780 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:25.989298 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 50459.3us
I20260807 19:49:25.994514 8778 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:25.936753 11081 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:25.948695 10364 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:25.991437 8740 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:25.995471 8849 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:25.950034 10950 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
W20260807 19:49:26.008945 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:25.982683 11105 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:26.017534 11105 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:26.018116 11105 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:26.014158 11102 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
W20260807 19:49:26.020409 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.017369 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:26.025662 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.031260 11061 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:26.033350 10889 raft_consensus.cc:3055] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:26.035550 10973 raft_consensus.cc:3055] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
W20260807 19:49:26.038105 11093 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.102s user 0.001s sys 0.000s
I20260807 19:49:26.039983 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:26.042287 8740 connection.cc:256] RPC call timeout handler was delayed by -1.2603703220374882s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
W20260807 19:49:26.076455 8740 connection.cc:256] RPC call timeout handler was delayed by -1.208170322002843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260807 19:49:26.076545 8911 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:25.947494 10977 consensus_queue.cc:260] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.068799 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.071614 8908 raft_consensus.cc:1240] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:26.074297 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.081928 8903 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.012094 11076 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.079317 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.087643 8906 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.073092 10889 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
W20260807 19:49:26.087867 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 103642us
I20260807 19:49:26.043203 11070 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:26.092245 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.080237 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:26.090348 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.027277 10974 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.097858 8902 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.072279 10556 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:26.012735 10780 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:26.068445 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 135557us
I20260807 19:49:26.072789 10973 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:26.097626 11070 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:26.122566 10889 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.123999 10889 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.132787 11116 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260807 19:49:26.135902 11101 log.cc:927] Time spent T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.263s user 0.001s sys 0.000s
I20260807 19:49:26.139237 11114 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:26.107745 11081 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.120915 11055 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:26.103524 11060 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.181228 10705 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:26.199559 10705 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:26.147318 10781 consensus_meta.cc:287] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.544s user 0.005s sys 0.004s
I20260807 19:49:26.200961 10781 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.201854 10705 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.147732 10973 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:25.930757 9888 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:26.152253 10780 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.072566 11061 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.152887 11082 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:26.120543 11070 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.122211 11118 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:26.165274 8902 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:26.163075 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:49:26.228380 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.181072 8903 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
W20260807 19:49:26.203701 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.204612 8913 raft_consensus.cc:2393] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:26.203181 10781 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.200019 11060 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.140043 11000 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.166148 11076 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:26.167209 11116 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:26.229549 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.182817 11114 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:26.246114 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:26.247121 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 170133us
W20260807 19:49:26.247910 8740 connection.cc:256] RPC call timeout handler was delayed by -1.2036703219637275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260807 19:49:26.230990 11118 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:26.250828 8906 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:26.249512 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:26.251919 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.247416 11114 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.245695 11116 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:26.249336 10973 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.258510 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.266623 9015 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.263442 10935 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:26.269311 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:26.274009 8778 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:26.285395 8790 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.252887 11118 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:26.289911 8740 connection.cc:256] RPC call timeout handler was delayed by -1.1892703219782561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
W20260807 19:49:26.290683 8740 connection.cc:256] RPC call timeout handler was delayed by -1.1757703218609095s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
W20260807 19:49:26.292016 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.302940 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:26.253393 11076 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.310802 8783 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.311993 8789 raft_consensus.cc:2410] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.315771 8968 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:26.282502 10935 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.326922 11090 consensus_queue.cc:1243] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer 671d8754170646eaac5748745eba58b4 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 19:49:26.303100 10358 consensus_queue.cc:1059] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
W20260807 19:49:26.342789 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 69938.7us
W20260807 19:49:26.345292 8740 connection.cc:256] RPC call timeout handler was delayed by -1.1541703219991177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
W20260807 19:49:26.345781 8740 connection.cc:256] RPC call timeout handler was delayed by -1.1469703218899667s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
W20260807 19:49:26.346076 8740 connection.cc:256] RPC call timeout handler was delayed by -1.1451703219208866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
W20260807 19:49:26.346524 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.347647 8740 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.348511 11104 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:26.349200 11104 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.350163 11090 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.351161 8778 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:26.333258 11073 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:26.355191 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.356488 8903 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.323665 10829 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260807 19:49:26.373801 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.374517 8906 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:26.372458 11116 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.357962 10692 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
W20260807 19:49:26.381456 11124 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
I20260807 19:49:26.382907 8911 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:26.385932 9044 tablet_copy_source_session.cc:215] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:49:26.387830 11116 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.389631 10990 raft_consensus.cc:3055] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:26.390060 10990 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:26.390911 10990 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.397106 10692 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:26.397675 10692 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:26.398149 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.398947 10692 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.400811 10935 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.401698 10895 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:26.390090 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 52637.9us
I20260807 19:49:26.404860 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.411049 8906 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:26.412695 9850 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:26.419137 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:26.419713 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.392156 10990 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.438392 8903 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.449415 8906 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:26.464178 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 65190.5us
W20260807 19:49:26.465821 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.423503 10935 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:26.477327 8903 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260807 19:49:26.420966 11073 raft_consensus.cc:697] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:26.456104 10692 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.428177 11061 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:26.486189 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.487315 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.488148 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.488951 9018 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:26.490787 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.491691 11090 consensus_queue.cc:1059] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:26.492897 11116 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.493109 11090 raft_consensus.cc:3055] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:26.493665 11090 raft_consensus.cc:740] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:26.494555 11073 consensus_queue.cc:237] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.495054 11090 consensus_queue.cc:260] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.479307 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.496697 11090 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:26.468219 11000 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.507968 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.508695 8902 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.511220 11104 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:26.512310 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:26.475716 11143 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:26.515026 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.513219 10988 raft_consensus.cc:993] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
W20260807 19:49:26.515890 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.522320 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.525177 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:26.525729 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.525938 8908 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.537616 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.497189 9019 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
W20260807 19:49:26.549780 11137 log.cc:927] Time spent T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.197s user 0.004s sys 0.003s
I20260807 19:49:26.554916 9044 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:37380 (request call id 85) took 1233 ms. Trace:
I20260807 19:49:26.555394 9044 rpcz_store.cc:276] 0807 19:49:25.343086 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:25.413206 (+ 70120us) service_pool.cc:224] Handling call
0807 19:49:26.385415 (+972209us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400418554. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b60e7d7 00007f68805d5e46 00007f68805df84e 00007f6877e5361b 00007f68805df7ca 00007f68805da8dc 00007f68805d5a8b 00007f68805ca6e1 00007f687f67d5d4 00007f687f67d562
0807 19:49:26.543352 (+157937us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762622848}
W20260807 19:49:26.544140 11105 consensus_meta.cc:287] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.526s user 0.013s sys 0.000s
I20260807 19:49:26.562969 8910 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.496671 11114 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.560554 8901 raft_consensus.cc:1240] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:26.540928 11076 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.546149 11085 raft_consensus.cc:1081] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Signalling peer 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:49:26.567549 8902 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:26.544508 11104 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.576107 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.579970 8910 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:26.580317 8902 raft_consensus.cc:1240] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:26.581079 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.560669 11107 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.583681 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.584125 9850 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:26.584524 9020 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.591102 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.597224 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.598500 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.599503 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.521420 11143 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.604772 11138 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:26.605902 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.608039 11100 raft_consensus.cc:3055] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:26.588223 10741 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:26.638839 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.585992 10935 raft_consensus.cc:697] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:26.622225 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.590863 10996 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:26.645393 8913 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.647686 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.619349 11100 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:26.626255 11107 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:26.649273 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.651909 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.574622 11105 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.637826 10741 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:26.658218 8911 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:26.645092 9018 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:26.666450 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.656385 10741 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:26.656780 11116 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.646538 10996 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.674616 11105 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:26.672633 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.676542 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:26.677845 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.666143 10977 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.713577 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.669514 11138 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:26.679699 11158 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.076s user 0.000s sys 0.001s
I20260807 19:49:26.716881 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.717279 11104 consensus_queue.cc:1059] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260807 19:49:26.722605 11143 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.733978 8740 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.738416 11134 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:26.739126 11104 raft_consensus.cc:3055] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:26.739529 11104 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:26.740322 11104 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.741547 11104 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.727684 11118 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.712999 11107 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:26.748059 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.753042 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:26.753136 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.754863 8908 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:26.755887 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.758183 9021 raft_consensus.cc:2393] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:26.763780 10996 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:26.765035 11130 log.cc:927] Time spent T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.518s user 0.004s sys 0.004s
W20260807 19:49:26.772673 11148 log.cc:927] Time spent T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.186s user 0.001s sys 0.000s
I20260807 19:49:26.732733 11114 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.718276 10973 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:26.680629 10935 consensus_queue.cc:237] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:26.774794 11019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a898110875ba40ab8e512e124aff4fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:49:26.775875 9022 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:26.777069 9015 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:49:26.789239 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 55340.3us
I20260807 19:49:26.791764 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.792514 8911 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:26.793040 8902 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:26.793633 11070 consensus_meta.cc:287] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.563s user 0.008s sys 0.008s
I20260807 19:49:26.793962 8908 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.794196 11070 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.795033 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.795622 8915 raft_consensus.cc:2376] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:26.799418 8775 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.800393 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.801239 8901 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.801749 11114 consensus_queue.cc:1059] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:26.802024 10973 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:26.800447 8783 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.803972 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:26.805497 8903 raft_consensus.cc:2410] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.802699 11138 raft_consensus.cc:3055] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:26.807741 11138 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:26.807953 8787 raft_consensus.cc:1275] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.810302 11114 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:26.810899 11114 raft_consensus.cc:697] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:26.811841 11114 consensus_queue.cc:237] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:26.813572 11161 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20260807 19:49:26.810449 11138 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:26.817281 8778 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.822815 8913 raft_consensus.cc:2410] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:26.681282 11100 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:26.813508 10889 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.651s user 0.014s sys 0.000s
I20260807 19:49:26.820181 8906 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:26.256072 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:26.837594 8778 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:26.841112 9017 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:26.841974 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.800750 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:26.847080 8787 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260807 19:49:26.837502 11138 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:26.825899 11118 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.820698 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:26.863956 8903 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.866755 8906 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.867859 8787 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:26.853788 9890 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:26.866981 8968 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:26.873946 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.874460 8913 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:26.837083 11105 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260807 19:49:26.878983 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.880172 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.881735 8906 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:26.883168 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.882642 8913 raft_consensus.cc:2393] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:26.885814 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.885977 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.886592 10358 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:26.887247 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.887395 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.855156 10782 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260807 19:49:26.871349 10990 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260807 19:49:26.889556 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:26.890758 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:26.891882 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:26.886500 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.888245 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:26.894296 9015 raft_consensus.cc:2376] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3. Re-sending same reply.
I20260807 19:49:26.894491 8901 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:26.895391 8915 raft_consensus.cc:1275] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.895643 8790 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:26.899050 8783 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:26.857561 8776 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:26.907064 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.907810 9022 raft_consensus.cc:2376] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2. Re-sending same reply.
I20260807 19:49:26.908828 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.909497 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.910692 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.913444 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:26.913343 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.915751 8901 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:26.920181 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:26.920909 8915 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:26.889084 11070 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:26.895176 9888 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33452 (request call id 157) took 1057 ms. Trace:
I20260807 19:49:26.925107 8790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:51024
I20260807 19:49:26.939466 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:26.935978 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.939630 8790 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:49:26.940759 8790 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:26.931293 8906 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:26.941161 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:26.943182 9022 raft_consensus.cc:2376] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3. Re-sending same reply.
W20260807 19:49:26.944242 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:26.960481 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:26.976466 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:26.989104 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 84627.1us
I20260807 19:49:26.989885 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:26.922842 11090 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.004045 11110 consensus_queue.cc:1048] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:27.006381 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:27.007185 11077 consensus_queue.cc:1059] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260807 19:49:27.007586 10384 tablet_copy_client.cc:538] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:27.014788 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:27.024959 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:27.026374 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:26.941048 11114 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:26.944566 10358 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:26.979928 11118 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.034962 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:27.027343 8790 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.039143 8783 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
W20260807 19:49:27.039469 9838 leader_election.cc:336] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.201s (SENT)
I20260807 19:49:26.993458 10973 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:26.994128 11107 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:26.995564 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:27.047003 8911 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:26.942163 9888 rpcz_store.cc:276] 0807 19:49:25.817935 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:25.818623 (+ 688us) service_pool.cc:224] Handling call
0807 19:49:25.852959 (+ 34336us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:25.869214 (+ 16255us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:26.649774 (+780560us) log.cc:844] Serialized 14 byte log entry
0807 19:49:26.851159 (+201385us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:26.853562 (+ 2403us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:26.859155 (+ 5593us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:26.862169 (+ 3014us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:26.875647 (+ 13478us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7084,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":10470,"thread_start_us":148587,"threads_started":1,"wal-append.queue_time_us":238887}
I20260807 19:49:27.054831 10358 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.055401 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:26.980054 11085 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260807 19:49:27.048189 8906 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:27.059262 8910 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:27.060297 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.058329 8911 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:27.062599 8902 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:27.049962 8790 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:27.055251 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.061223 8787 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:27.075222 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:27.049995 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:27.048862 9838 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:26.992585 10741 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:27.054836 11164 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
W20260807 19:49:27.136147 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 145862us
W20260807 19:49:27.137182 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.024520 11090 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.139739 8913 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260807 19:49:27.145861 8913 pending_rounds.cc:85] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 1
I20260807 19:49:27.146184 8913 pending_rounds.cc:107] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260807 19:49:27.146466 8913 raft_consensus.cc:2889] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 19:49:27.140921 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:27.147799 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.149755 9889 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260807 19:49:27.150252 9889 pending_rounds.cc:85] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Aborting all ops after (but not including) 5
I20260807 19:49:27.150581 9889 pending_rounds.cc:107] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260807 19:49:27.150851 9889 raft_consensus.cc:2889] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:27.152815 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.153554 8787 raft_consensus.cc:2376] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:27.154498 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.155223 8787 raft_consensus.cc:2393] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:27.156047 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.156868 8787 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:27.157594 11162 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.163s user 0.000s sys 0.000s
I20260807 19:49:27.157812 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.159762 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.161067 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:27.161181 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 138499us
W20260807 19:49:27.162017 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.162652 11000 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:27.169178 8789 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:27.172011 11143 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:49:27.173381 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.174284 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.174811 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.175489 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.176234 8789 raft_consensus.cc:2393] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:27.177053 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.178099 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.179248 8968 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:27.180279 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.182025 8911 raft_consensus.cc:2376] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4. Re-sending same reply.
I20260807 19:49:27.131738 10781 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.048483 11118 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.194598 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.196640 8911 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:27.201568 8789 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:27.206344 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.208565 8789 raft_consensus.cc:2393] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:27.210346 8788 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:27.052896 11114 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.057000 10973 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.214911 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:27.147063 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:27.210073 10756 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260807 19:49:27.219492 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:27.220198 8737 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:27.225123 9022 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:27.220755 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.225943 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.226696 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.227432 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.169284 10765 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.056870 11107 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.190244 10741 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:27.196810 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.228586 10977 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:27.244477 8789 raft_consensus.cc:2410] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:27.194736 10860 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.245667 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:27.246851 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.256702 8908 raft_consensus.cc:2376] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:27.250205 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.250883 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.221355 11090 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:27.258440 8901 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:27.259276 9022 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:27.233623 11065 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:27.276630 8787 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:27.235981 9888 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:27.211215 11082 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260807 19:49:27.234354 11104 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.243098 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:27.305863 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.219520 10915 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260807 19:49:27.212985 10781 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.246994 10988 raft_consensus.cc:3055] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:27.254547 10977 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:27.341406 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.342139 8913 raft_consensus.cc:2376] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:27.228577 11089 consensus_queue.cc:1059] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260807 19:49:27.221419 11073 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260807 19:49:27.281071 11065 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:27.350208 8915 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:27.247973 10765 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:27.339330 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:27.343524 10988 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:27.350677 10977 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.388046 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.394832 8901 raft_consensus.cc:2376] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4. Re-sending same reply.
I20260807 19:49:27.344878 8904 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:27.280014 10860 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.391196 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:27.396399 8915 raft_consensus.cc:2376] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5. Re-sending same reply.
I20260807 19:49:27.357292 11065 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.340503 11104 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.401257 8915 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:27.408299 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:27.458402 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:27.531206 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:27.464561 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.411206 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.544034 8913 raft_consensus.cc:2410] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260807 19:49:27.515283 11165 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.377s user 0.000s sys 0.001s
I20260807 19:49:27.522899 8776 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:27.559340 8915 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:49:27.522717 10765 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:27.521270 10973 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:27.408291 11118 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:27.576568 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.441048 11090 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:27.579982 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:27.583635 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 404447us
I20260807 19:49:27.594398 8968 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:27.601560 9850 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:27.602365 11169 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.599s user 0.005s sys 0.000s
I20260807 19:49:27.604696 10782 raft_consensus.cc:3055] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:27.605283 10782 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:27.607146 8779 raft_consensus.cc:1275] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:27.607631 8779 pending_rounds.cc:85] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:27.608476 8779 pending_rounds.cc:107] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:27.608711 8779 raft_consensus.cc:2889] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:27.578089 11114 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:27.609454 10765 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:49:27.610553 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.620114 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:27.621582 8780 raft_consensus.cc:1275] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:27.606379 10782 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.621975 8780 pending_rounds.cc:85] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:27.622283 11072 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:27.622648 8780 pending_rounds.cc:107] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:27.622928 8780 raft_consensus.cc:2889] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:27.623103 11100 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.624432 8780 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 19:49:27.631016 8776 raft_consensus.cc:1275] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:27.616166 10765 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:27.639132 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:27.639880 8915 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:27.639701 11086 raft_consensus.cc:3055] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:27.639191 10765 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:27.609035 10741 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260807 19:49:27.593865 11090 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.658685 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:27.667704 8913 raft_consensus.cc:2376] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4. Re-sending same reply.
I20260807 19:49:27.615141 11000 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:27.556860 10988 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.658238 11104 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:27.661825 11176 log.cc:927] Time spent T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.264s user 0.001s sys 0.000s
W20260807 19:49:27.676951 11170 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.679s user 0.006s sys 0.001s
W20260807 19:49:27.647176 11174 log.cc:927] Time spent T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.260s user 0.001s sys 0.000s
I20260807 19:49:27.633337 11107 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:27.647764 11181 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:27.680464 11082 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 19:49:27.646667 11086 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:27.670269 10781 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:27.682593 10860 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:27.701766 10977 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.702479 9888 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:27.709479 11086 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:27.714421 11019 consensus_meta.cc:287] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.509s user 0.008s sys 0.002s
I20260807 19:49:27.690656 11000 raft_consensus.cc:697] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:27.731132 8789 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:27.729460 8737 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:27.735656 8780 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260807 19:49:27.736053 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.736928 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.740868 8780 pending_rounds.cc:85] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:27.741230 8780 pending_rounds.cc:107] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:27.741503 8780 raft_consensus.cc:2889] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:27.691285 11181 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:27.745872 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 96288.2us
W20260807 19:49:27.724925 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 110025us
I20260807 19:49:27.740161 11107 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:27.738835 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.727276 10765 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.790081 8790 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:27.786015 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:27.797891 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:27.805498 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 82563.5us
W20260807 19:49:27.806411 8849 leader_election.cc:336] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.555s (SENT)
I20260807 19:49:27.809777 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.810915 8849 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:27.811120 8790 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.799660 11107 raft_consensus.cc:697] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:27.817391 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 57) took 1001 ms. Trace:
I20260807 19:49:27.817952 8788 rpcz_store.cc:276] 0807 19:49:26.815909 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:26.816032 (+ 123us) service_pool.cc:224] Handling call
0807 19:49:26.997602 (+181570us) spinlock_profiling.cc:243] Waited 7.89 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:26.997822 (+ 220us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:27.210791 (+212969us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:27.817150 (+606359us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540008bbd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:27.817328 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":506061440}
I20260807 19:49:27.817366 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:27.821678 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 48107.8us
I20260807 19:49:27.810576 11019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:33452 (request call id 150) took 2959 ms. Trace:
W20260807 19:49:27.823467 8849 leader_election.cc:336] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.589s (SENT)
I20260807 19:49:27.824361 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:27.827600 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.848567 8849 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:27.829000 10765 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:27.829859 10977 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:27.831163 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:27.831910 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:27.832568 10781 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:27.833588 9019 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:27.833472 10384 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:27.856678 10358 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260807 19:49:27.862393 11180 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:27.840601 11107 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.857999 11085 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260807 19:49:27.867003 10765 consensus_queue.cc:1048] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:27.845527 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:27.814182 11197 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:27.868731 11197 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:27.869179 11197 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:27.869506 10781 consensus_queue.cc:1059] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:27.870472 10781 raft_consensus.cc:3055] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:27.870822 10781 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:27.858335 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.798743 11000 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.859177 10996 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260807 19:49:27.842047 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.846489 11019 rpcz_store.cc:276] 0807 19:49:24.846635 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.848499 (+ 1864us) service_pool.cc:224] Handling call
0807 19:49:25.109158 (+260659us) ts_tablet_manager.cc:934] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
0807 19:49:26.580108 (+1470950us) tablet_copy_client.cc:368] Tablet copy session begun
0807 19:49:27.204785 (+624677us) tablet_metadata.cc:665] Metadata flushed
0807 19:49:27.204955 (+ 170us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 19:49:27.730875 (+525920us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 19:49:27.765433 (+ 34558us) tablet_replica.cc:206] Creating consensus instance
0807 19:49:27.800058 (+ 34625us) ts_tablet_manager.cc:989] Replica registered.
0807 19:49:27.806343 (+ 6285us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":35620,"tablet-copy.queue_time_us":223624,"thread_start_us":217388,"threads_started":2}
I20260807 19:49:27.846164 11090 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:27.875152 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.875345 8776 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:27.875976 8787 raft_consensus.cc:2393] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:27.876474 8775 raft_consensus.cc:2410] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:27.877565 11019 tablet_copy_client.cc:806] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 0 data blocks...
I20260807 19:49:27.878219 11019 tablet_copy_client.cc:670] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Starting download of 1 WAL segments...
I20260807 19:49:27.878520 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.879429 8778 raft_consensus.cc:2410] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:27.882588 11055 raft_consensus.cc:3055] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:27.883091 11055 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:27.884074 11055 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.885205 11055 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.888200 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:27.860862 8849 leader_election.cc:336] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.633s (SENT)
I20260807 19:49:27.844237 11181 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:27.846593 11196 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260807 19:49:27.871557 8788 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:27.857285 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.871407 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:27.871837 11065 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.828068 8790 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:27.894753 8780 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:27.886150 9889 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:27.894021 11141 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:27.894840 11110 raft_consensus.cc:3055] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:27.909562 11110 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:27.911885 11065 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:27.913060 8790 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:27.895953 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.914570 8787 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:27.915608 8737 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:27.910254 10996 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:27.916242 10996 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:27.916713 10996 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:27.917451 8628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38426 (request call id 206) took 5962 ms. Trace:
I20260807 19:49:27.897590 8788 raft_consensus.cc:1275] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:27.876477 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.916715 8787 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:27.916242 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 62) took 1065 ms. Trace:
I20260807 19:49:27.925244 8789 rpcz_store.cc:276] 0807 19:49:26.850396 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:27.607675 (+757279us) service_pool.cc:224] Handling call
0807 19:49:27.607965 (+ 290us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:27.731631 (+123666us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:27.915993 (+184362us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54000898d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:27.916160 (+ 167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282822912}
I20260807 19:49:27.923666 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:27.924569 11100 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:27.926942 11100 raft_consensus.cc:697] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:27.927968 11100 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:27.929240 11141 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.930418 9893 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:27.896881 10781 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.931152 8780 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:27.931571 11065 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:27.932894 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 60) took 1083 ms. Trace:
I20260807 19:49:27.925902 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:27.940457 8783 rpcz_store.cc:276] 0807 19:49:26.849792 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:27.338252 (+488460us) service_pool.cc:224] Handling call
0807 19:49:27.886414 (+548162us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:27.886561 (+ 147us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:27.886577 (+ 16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:27.886607 (+ 30us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:27.886624 (+ 17us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:27.886661 (+ 37us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:27.932686 (+ 46025us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b540008b6d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:27.932807 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229138304}
W20260807 19:49:27.942704 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 81870.9us
I20260807 19:49:27.910444 11110 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:27.945387 11110 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.948402 8968 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:27.949962 9838 leader_election.cc:336] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.788s (SENT)
I20260807 19:49:27.950472 9838 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:27.951642 11000 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:27.952699 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.954725 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:27.955349 8789 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.956228 11181 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:27.944085 8849 leader_election.cc:336] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.671s (SENT)
W20260807 19:49:27.959700 8849 leader_election.cc:336] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.774s (SENT)
I20260807 19:49:27.940760 8778 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:27.917826 8628 rpcz_store.cc:276] 0807 19:49:21.955244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:22.017276 (+ 62032us) service_pool.cc:224] Handling call
0807 19:49:27.917383 (+5900107us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:25.125102 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:25.133278 (+ 8176us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:25.133344 (+ 66us) write_op.cc:435] Acquired schema lock
0807 19:49:25.133362 (+ 18us) tablet.cc:662] Decoding operations
0807 19:49:25.136758 (+ 3396us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:25.136885 (+ 127us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:25.136911 (+ 26us) tablet.cc:685] Acquiring locks for 10 operations
0807 19:49:25.137735 (+ 824us) tablet.cc:701] Row locks acquired
0807 19:49:25.137757 (+ 22us) write_op.cc:260] PREPARE: finished
0807 19:49:25.137953 (+ 196us) write_op.cc:270] Start()
0807 19:49:25.138105 (+ 152us) write_op.cc:276] Timestamp: P: 1786132165137887 usec, L: 0
0807 19:49:25.138126 (+ 21us) op_driver.cc:348] REPLICATION: starting
0807 19:49:25.138844 (+ 718us) log.cc:844] Serialized 3999 byte log entry
0807 19:49:25.347062 (+208218us) op_driver.cc:464] REPLICATION: finished
0807 19:49:25.809568 (+462506us) write_op.cc:301] APPLY: starting
0807 19:49:25.810736 (+ 1168us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:25.842760 (+ 32024us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:25.869412 (+ 26652us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:25.982092 (+112680us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:26.014559 (+ 32467us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0807 19:49:26.014621 (+ 62us) write_op.cc:312] APPLY: finished
0807 19:49:26.014969 (+ 348us) log.cc:844] Serialized 76 byte log entry
0807 19:49:26.015166 (+ 197us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:26.015355 (+ 189us) write_op.cc:454] Released schema lock
0807 19:49:26.015414 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:26.015874 (+ 460us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:26.015904 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5304,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":404651,"apply.run_cpu_time_us":5369,"apply.run_wall_time_us":211971,"num_ops":10,"prepare.queue_time_us":214170,"prepare.run_cpu_time_us":23317,"prepare.run_wall_time_us":78926,"raft.queue_time_us":94975,"raft.run_cpu_time_us":25082,"raft.run_wall_time_us":320018,"replication_time_us":179439,"thread_start_us":262693,"threads_started":4,"wal-append.queue_time_us":51691,"wal-append.run_cpu_time_us":11281,"wal-append.run_wall_time_us":1827890}]]}
I20260807 19:49:27.963680 8903 raft_consensus.cc:1217] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2] Dedup: 2.2->[]
I20260807 19:49:27.964298 8903 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:27.951709 10782 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.971477 11065 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:27.972425 11065 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:27.972779 11065 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:27.962603 11196 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
W20260807 19:49:27.974893 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:27.975951 8789 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:27.977571 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 51796us
I20260807 19:49:27.971815 11082 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260807 19:49:27.986958 10781 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:28.001336 10358 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 19:49:28.005888 8849 leader_election.cc:336] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.827s (SENT)
I20260807 19:49:28.006040 11217 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:27.974098 11188 consensus_queue.cc:1059] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260807 19:49:28.006693 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:27.990296 11090 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.010248 11202 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:27.993963 11141 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:28.023528 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.013173 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.031461 8968 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:28.016448 11090 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.042666 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:28.044212 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:28.047012 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:28.050334 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.057770 8740 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:28.069648 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 91307.3us
I20260807 19:49:28.048956 11107 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.051923 10977 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:28.063234 11114 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:28.052027 11104 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:28.079849 8972 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:28.066289 11118 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:28.075980 10756 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:28.076314 10973 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260807 19:49:28.094152 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.104972 10765 raft_consensus.cc:3055] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:28.105695 10765 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:28.106791 9027 raft_consensus.cc:1275] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:49:28.106664 10765 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:28.107889 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.107571 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.113061 11226 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.114208 11155 consensus_queue.cc:1048] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260807 19:49:28.107446 11215 log.cc:927] Time spent T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.077s user 0.001s sys 0.000s
I20260807 19:49:28.146503 8972 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:28.148515 10358 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:28.148494 9838 leader_election.cc:336] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.985s (SENT)
W20260807 19:49:28.149626 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.160842 8779 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:28.161346 8779 pending_rounds.cc:85] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:28.161590 8779 pending_rounds.cc:107] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:28.161829 8779 raft_consensus.cc:2889] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:28.092808 11104 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.166158 8775 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:28.166919 11104 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:28.171674 11077 consensus_queue.cc:1175] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "671d8754170646eaac5748745eba58b4" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260807 19:49:28.149849 9838 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:28.183405 8849 leader_election.cc:336] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.056s (SENT)
I20260807 19:49:28.183939 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.115356 11090 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:28.188196 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.190162 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.161708 11221 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260807 19:49:28.190665 11019 tablet_copy_client.cc:538] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:49:28.116639 10756 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:28.193086 11089 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:28.193303 8972 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:28.205094 8849 leader_election.cc:336] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.063s (SENT)
I20260807 19:49:28.147881 10765 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:28.228938 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 155213us
I20260807 19:49:28.231473 8740 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:28.232990 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:28.100158 11230 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260807 19:49:28.178038 11197 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.244236 8788 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:28.244885 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 51668us
I20260807 19:49:28.197008 11155 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:28.256472 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1007) took 1296 ms. Trace:
I20260807 19:49:28.257747 8779 rpcz_store.cc:276] 0807 19:49:26.959731 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:27.614245 (+654514us) service_pool.cc:224] Handling call
0807 19:49:28.160419 (+546174us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:28.162404 (+ 1985us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:28.256163 (+ 93759us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b540031e050. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:28.256390 (+ 227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1247271296}
I20260807 19:49:28.213340 11000 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:28.178349 11073 consensus_queue.cc:1059] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260807 19:49:28.256909 10358 consensus_queue.cc:1059] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:28.206540 10950 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:28.206288 11090 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.196276 11085 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260807 19:49:28.213593 11114 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:28.267750 8788 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:28.270429 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 65370.2us
W20260807 19:49:28.291566 8849 leader_election.cc:336] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.070s (SENT)
I20260807 19:49:28.291955 8849 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:28.286770 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.266614 8972 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:28.305135 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1106) took 1141 ms. Trace:
I20260807 19:49:28.305692 8788 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:28.245268 11110 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 19:49:28.311396 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 44871.9us
I20260807 19:49:28.256561 10364 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:28.253726 11118 raft_consensus.cc:697] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:28.268503 10977 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:28.283215 11081 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:28.346437 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 54884.2us
W20260807 19:49:28.356719 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.318545 10364 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:28.338505 11141 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:28.377759 11082 raft_consensus.cc:3055] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:28.378098 11244 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:28.378252 11082 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:28.377597 11141 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.378690 11244 raft_consensus.cc:697] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:28.379741 11244 consensus_queue.cc:237] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.338335 11055 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.373862 11118 consensus_queue.cc:237] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.315968 11196 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260807 19:49:28.290596 11090 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.344075 11199 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.380379 10996 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.381484 10895 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:28.422101 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 189185us
W20260807 19:49:28.424364 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 110862us
I20260807 19:49:28.345119 11081 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.416749 11234 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:28.437615 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.438311 8901 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:28.429368 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.445552 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.442452 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.367846 10756 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.446720 8911 raft_consensus.cc:2376] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5. Re-sending same reply.
W20260807 19:49:28.449436 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.452867 8901 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:28.454303 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.440785 10977 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.440393 10781 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.462478 10781 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.462844 10996 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:28.426707 10895 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.468312 11257 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:28.469143 11239 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260807 19:49:28.445159 11230 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.452693 11234 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.475505 11202 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.476428 11202 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.430703 11065 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
W20260807 19:49:28.486384 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 57092.5us
I20260807 19:49:28.490617 11065 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:28.487272 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.428880 11114 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.441660 10364 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.525121 10364 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:28.525635 10895 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.526197 10915 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.526741 10915 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.533044 8913 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:28.536942 10384 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260807 19:49:28.537647 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:28.543576 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 56390.7us
I20260807 19:49:28.545147 8913 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:28.546108 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.499740 11197 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:28.458539 11082 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:28.549199 11249 log.cc:927] Time spent T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.115s user 0.000s sys 0.000s
I20260807 19:49:28.549638 11082 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:28.485929 11234 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:28.553913 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.485421 11230 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.561854 11230 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:28.561981 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.564666 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.563097 11255 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.571352 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.573719 8915 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:28.576578 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.578816 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.584739 11247 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:28.585181 11237 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.579008 11118 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.586009 11256 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.577323 10988 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:28.586797 11254 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.587984 10895 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.588680 10895 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.590687 10384 tablet_bootstrap.cc:492] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:28.591195 10895 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.591619 10384 ts_tablet_manager.cc:1403] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 0.759s user 0.022s sys 0.014s
W20260807 19:49:28.589141 11245 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.143s user 0.000s sys 0.002s
I20260807 19:49:28.594481 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.592599 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.585397 10915 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:28.582897 8913 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:28.588968 9019 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:28.569648 11143 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.569882 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.610981 8903 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:28.623778 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.631855 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.633018 8913 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:28.640945 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 46546.2us
I20260807 19:49:28.644392 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.645206 8913 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:28.651922 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.653057 8740 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.624924 11090 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.642185 11118 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:28.668773 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.660444 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.665848 11114 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.659891 11254 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.586573 11256 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.636845 11143 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.685627 11234 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.686487 11227 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:28.687089 11227 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:28.689335 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.690423 11234 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.691859 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:28.689527 11265 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.110s user 0.000s sys 0.001s
I20260807 19:49:28.675801 11090 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.700799 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.692813 11234 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:28.702243 10895 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.700039 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.705160 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.705844 11143 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.707168 9021 raft_consensus.cc:1275] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:28.708511 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.708894 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:28.673234 10860 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260807 19:49:28.720050 8903 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:28.721345 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.725132 11254 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.732301 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.733042 9015 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:28.735725 8856 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:28.735667 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.737178 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:28.741050 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.682901 9008 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:28.692317 10781 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.745935 9020 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:28.694193 11081 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.719697 11234 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:28.752256 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56357.3us
I20260807 19:49:28.752499 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.755103 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.756175 8903 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:28.764884 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.766247 8903 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:28.688076 11227 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.693956 10384 raft_consensus.cc:359] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5 [term 2 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.748097 11199 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:28.785292 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.719921 10895 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260807 19:49:28.733436 11143 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260807 19:49:28.726112 11254 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.771000 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:28.775259 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.783002 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.305588 8902 rpcz_store.cc:276] 0807 19:49:27.164081 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:27.165818 (+ 1737us) service_pool.cc:224] Handling call
0807 19:49:27.166114 (+ 296us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:27.195037 (+ 28923us) raft_consensus.cc:1534] Early marking committed up to index 2
0807 19:49:27.596289 (+401252us) log.cc:844] Serialized 14 byte log entry
0807 19:49:28.160985 (+564696us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b18003232c0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a342a 00007f687b60ee0a
0807 19:49:28.304837 (+143852us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:28.304891 (+ 54us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:28.304932 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:28.304989 (+ 57us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:28.305081 (+ 92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5507,"prepare.queue_time_us":389191,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":28279,"spinlock_wait_cycles":8831360,"thread_start_us":480990,"threads_started":2}
W20260807 19:49:28.802234 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.803134 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.805471 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.805811 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.806079 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.810413 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:28.811121 8788 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 4
I20260807 19:49:28.812247 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.813009 8789 raft_consensus.cc:2393] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:28.817873 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:28.818073 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.818729 8913 raft_consensus.cc:2393] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:28.782871 11199 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:28.824705 8903 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:28.835021 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:28.765192 10860 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260807 19:49:28.796113 11227 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:28.843452 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.836858 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.792191 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.838738 8903 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:28.755105 11234 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.845134 8778 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:28.792129 11081 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260807 19:49:28.849551 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.850970 8902 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:28.809321 11090 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.858593 8915 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:28.820463 11114 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.861655 11114 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.793524 10781 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
W20260807 19:49:28.864097 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:28.862370 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:28.808732 11107 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.867993 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.838971 11199 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:28.873477 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 76020.7us
I20260807 19:49:28.859035 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.891577 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:28.884001 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.893864 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.895733 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:28.897574 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.899008 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:28.856267 11256 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.900693 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:28.889592 8788 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:28.901117 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.877424 10384 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:28.882954 10895 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.901412 8780 raft_consensus.cc:2376] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3. Re-sending same reply.
I20260807 19:49:28.907138 8783 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:28.909224 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.867363 11090 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.910432 8788 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:28.912237 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:28.913275 8788 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:28.913589 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.872088 11107 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:28.901235 11118 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:28.929875 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.930702 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.933979 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:28.932801 8780 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:28.936722 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:28.939090 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 64029.4us
I20260807 19:49:28.909263 10977 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.935248 11247 consensus_queue.cc:1048] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 19:49:28.932797 11085 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:28.958552 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:28.960990 8780 raft_consensus.cc:2376] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4. Re-sending same reply.
I20260807 19:49:28.961934 11055 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:28.965306 11288 log.cc:927] Time spent T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.116s user 0.000s sys 0.001s
I20260807 19:49:28.947157 11104 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:28.943673 11255 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:28.900413 11298 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260807 19:49:28.964601 11055 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:28.971724 11086 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:28.949465 11237 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.981846 11254 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.952716 11244 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:28.947511 9018 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:28.944857 10384 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } attrs { replace: true } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:29.013226 8740 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:29.007593 11019 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap starting.
I20260807 19:49:29.025040 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:29.021803 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 238861us
I20260807 19:49:28.969794 11085 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.030049 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.017480 10988 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.022325 11114 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.023797 11118 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:29.036571 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.029204 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.028793 11254 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.028712 11141 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.042723 8849 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.039834 8856 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:29.032415 11237 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:29.047796 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.045804 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1039) took 1312 ms. Trace:
I20260807 19:49:29.054497 8790 rpcz_store.cc:276] 0807 19:49:27.733438 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:27.913856 (+180418us) service_pool.cc:224] Handling call
0807 19:49:27.921086 (+ 7230us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:27.922189 (+ 1103us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:27.922220 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:27.922672 (+ 452us) log.cc:844] Serialized 26 byte log entry
0807 19:49:28.786368 (+863696us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b18003e28a0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a32c1 00007f687b5bfdef
0807 19:49:28.842061 (+ 55693us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:28.869372 (+ 27311us) log.cc:844] Serialized 14 byte log entry
0807 19:49:28.874804 (+ 5432us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:28.874894 (+ 90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:29.033903 (+159009us) raft_consensus.cc:1685] finished
0807 19:49:29.045568 (+ 11665us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:29.045741 (+ 173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28647,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":8754,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":14870,"spinlock_wait_cycles":30928256,"thread_start_us":55494,"threads_started":1,"wal-append.queue_time_us":959235}
I20260807 19:49:29.071449 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.072631 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.073736 8849 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.075999 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.076264 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.042201 11255 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:29.077163 8903 raft_consensus.cc:2410] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260807 19:49:29.079967 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.038717 9019 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:29.079030 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.083547 8849 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.080991 8903 raft_consensus.cc:1275] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:29.082713 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.070854 11181 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:29.074877 10781 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:29.048714 11118 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:29.085402 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.092454 11316 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.093048 11237 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:29.090795 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:29.096860 11314 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:29.097308 11314 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.097669 11314 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:29.085743 8902 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:29.087998 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.102371 9021 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:29.070737 11180 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:29.103161 11180 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:29.103566 11180 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:29.038067 10988 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.077490 11141 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:29.105933 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.077838 11090 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.111258 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.084965 11254 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:29.079909 11306 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:29.088835 11181 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.104836 10895 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:29.109311 11316 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:29.135010 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:29.110337 11232 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.135864 11232 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.138468 11232 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:29.145339 11082 consensus_meta.cc:287] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.595s user 0.007s sys 0.007s
I20260807 19:49:29.108866 11255 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260807 19:49:29.147225 11255 raft_consensus.cc:3055] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:29.147634 11255 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:29.148567 11255 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.149983 11255 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:29.120081 11085 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:29.125723 10895 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.151754 10895 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:29.169059 11118 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.173273 8790 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:29.176607 9020 raft_consensus.cc:1275] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260807 19:49:29.178838 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:29.173305 11237 consensus_queue.cc:1175] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "671d8754170646eaac5748745eba58b4" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 19:49:29.106972 11143 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 19:49:29.122603 11114 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:29.125211 11181 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:29.110607 11260 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.113816 11247 consensus_queue.cc:1048] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260807 19:49:29.180294 10384 ts_tablet_manager.cc:1434] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.588s user 0.012s sys 0.003s
I20260807 19:49:29.196554 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:29.204421 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:29.205206 8790 raft_consensus.cc:2376] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4. Re-sending same reply.
W20260807 19:49:29.218313 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:29.219139 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:29.233949 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:29.235538 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:29.238525 8790 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:29.239429 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.241279 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:29.241910 8790 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:29.206146 10384 ts_tablet_manager.cc:1467] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Tablet startup took 606ms
I20260807 19:49:29.230345 9893 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:29.253538 10756 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.259696 8776 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:29.260032 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.260159 10756 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.261540 8780 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260807 19:49:29.275200 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 191730us
I20260807 19:49:29.276024 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.276793 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:29.285586 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 402379us
I20260807 19:49:29.286659 8790 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:29.268685 11114 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260807 19:49:29.290031 11318 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.103s user 0.000s sys 0.001s
I20260807 19:49:29.274385 11324 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.281486 10781 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.295454 11316 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.306677 10988 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:29.297688 11312 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 19:49:29.314450 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1209) took 1506 ms. Trace:
I20260807 19:49:29.307952 11324 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:29.332031 10756 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:29.355926 11107 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:29.366192 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 89471us
I20260807 19:49:29.367174 8849 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.368431 8849 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.370007 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.374624 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.377965 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.391465 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.392091 8913 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:29.393787 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.394600 8913 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:29.392391 8780 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:29.398432 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.399154 8780 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:29.400275 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.403352 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.404044 8780 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:29.405052 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.408212 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:29.408305 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 133271us
W20260807 19:49:29.250290 10384 ts_tablet_manager.cc:1469] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Trace:
0807 19:49:16.191677 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:16.195119 (+ 3442us) service_pool.cc:224] Handling call
0807 19:49:16.231885 (+ 36766us) ts_tablet_manager.cc:934] T 0ba47e79caa54c409c53304a66ce0160 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185)
0807 19:49:24.714208 (+8482323us) tablet_copy_client.cc:368] Tablet copy session begun
0807 19:49:24.819096 (+104888us) tablet_metadata.cc:665] Metadata flushed
0807 19:49:24.819243 (+ 147us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 19:49:24.957210 (+137967us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 19:49:24.957592 (+ 382us) tablet_replica.cc:206] Creating consensus instance
0807 19:49:24.958020 (+ 428us) ts_tablet_manager.cc:989] Replica registered.
0807 19:49:24.958173 (+ 153us) inbound_call.cc:177] Queueing success response
0807 19:49:25.268943 (+310770us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b18002c1820. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f6878477d1a 00007f6881c0b5d9 00007f68783a2aa8 00007f68784fd722 00007f68805adb33 00007f68805ad36d 00007f68806491e8 00007f688067ad5e 00007f688065fded
0807 19:49:27.832771 (+2563828us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0807 19:49:28.592257 (+759486us) ts_tablet_manager.cc:1435] Starting tablet replica
0807 19:49:28.601728 (+ 9471us) tablet_replica.cc:261] Starting instrumentation
0807 19:49:28.602466 (+ 738us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":61985,"mutex_wait_us":21729,"raft.queue_time_us":38986,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":13628,"spinlock_wait_cycles":9960704,"tablet-copy.queue_time_us":9313,"tablet-download-pool.queue_time_us":162,"tablet-download-pool.run_cpu_time_us":1920,"tablet-download-pool.run_wall_time_us":1940899,"thread_start_us":625082,"threads_started":1}
W20260807 19:49:29.449627 11327 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20260807 19:49:29.450961 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1139) took 1005 ms. Trace:
I20260807 19:49:29.373678 11141 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:29.393800 11060 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.371814 10781 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:29.371992 11316 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:29.433516 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.431782 11107 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:29.434909 11055 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:29.441236 11312 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.461452 11180 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.477355 8908 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:29.473179 10782 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.484572 11118 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.482029 11107 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.500061 8805 tablet_copy_service.cc:342] P 8e9ad2b5e2a94d4faff90af735c6d792: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-0ba47e79caa54c409c53304a66ce0160 received from {username='slave'} at 127.0.0.1:53512
I20260807 19:49:29.503366 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.506037 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:29.507067 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.510179 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.512023 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.512588 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.525245 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.525696 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.526433 8903 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:29.527868 8849 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.342008 8775 rpcz_store.cc:276] 0807 19:49:27.808272 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:28.170045 (+361773us) service_pool.cc:224] Handling call
0807 19:49:28.170673 (+ 628us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:28.171543 (+ 870us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:28.171574 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:28.936446 (+764872us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b2c0009a9c8. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6879effc71 00007f6879f0f5d2 00007f687b63fde1 00007f687b60481a 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22
0807 19:49:28.937083 (+ 637us) log.cc:844] Serialized 26 byte log entry
0807 19:49:28.951828 (+ 14745us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:28.951924 (+ 96us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:28.952181 (+ 257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:29.294699 (+342518us) raft_consensus.cc:1685] finished
0807 19:49:29.294727 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:29.312738 (+ 18011us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11283,"raft.queue_time_us":143347,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":13321,"spinlock_wait_cycles":29551488,"thread_start_us":14068,"threads_started":1,"wal-append.queue_time_us":141041}
I20260807 19:49:29.451357 8904 rpcz_store.cc:276] 0807 19:49:28.445798 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:28.819267 (+373469us) service_pool.cc:224] Handling call
0807 19:49:28.819752 (+ 485us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:28.956557 (+136805us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:28.956625 (+ 68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:28.976826 (+ 20201us) log.cc:844] Serialized 26 byte log entry
0807 19:49:29.316866 (+340040us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:29.318011 (+ 1145us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:29.336067 (+ 18056us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:29.450729 (+114662us) raft_consensus.cc:1685] finished
0807 19:49:29.450756 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:29.450901 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2914,"raft.queue_time_us":80710,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6262,"thread_start_us":465514,"threads_started":2,"wal-append.queue_time_us":393218}
I20260807 19:49:29.531041 8805 tablet_copy_service.cc:434] P 8e9ad2b5e2a94d4faff90af735c6d792: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-0ba47e79caa54c409c53304a66ce0160 on tablet 0ba47e79caa54c409c53304a66ce0160 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:29.485766 11312 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.534864 8908 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:29.535140 8849 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.536243 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:29.539659 8903 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:29.542162 8849 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.543654 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.545113 8849 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.548744 8911 raft_consensus.cc:1275] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:29.490592 10782 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.547253 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:29.550338 8849 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.553140 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.554687 11237 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:29.555152 11237 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:29.555233 10973 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:29.555675 11232 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:29.556279 10973 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:29.556773 10973 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:29.557096 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.557628 10973 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.558473 11237 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:29.559273 10973 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:29.573917 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:29.538396 11081 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.537756 11330 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.537719 11260 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.549185 11141 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:29.582381 11286 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.888s user 0.006s sys 0.000s
I20260807 19:49:29.540542 11060 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:29.548928 11316 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:29.507613 11118 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.577914 11181 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.552095 11143 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:29.533936 11187 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260807 19:49:29.601801 8968 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:29.530995 10895 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.575318 11232 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.573446 11332 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:29.551726 11065 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.059s
W20260807 19:49:29.614153 11104 consensus_meta.cc:287] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.634s user 0.013s sys 0.000s
I20260807 19:49:29.576557 11324 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.615145 11316 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.620409 11081 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.641454 8915 raft_consensus.cc:1240] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:29.624186 11357 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:29.625782 11060 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:29.630654 11196 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:29.614200 11260 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.631735 11141 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:29.651702 11343 log.cc:927] Time spent T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.078s user 0.001s sys 0.000s
I20260807 19:49:29.655347 11237 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.656898 11358 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:29.675966 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:29.639451 8904 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 19:49:29.657075 11312 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.657631 11356 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.657794 11143 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:29.661293 11118 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.662261 11107 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.661157 11000 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.634660 11104 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:29.632984 11352 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260807 19:49:29.640573 11316 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:29.629575 11232 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:29.705971 11255 consensus_meta.cc:287] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.555s user 0.008s sys 0.004s
W20260807 19:49:29.757586 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 226779us
I20260807 19:49:29.665827 11180 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.771179 11143 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:29.676090 11254 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:29.780841 11286 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.092s user 0.001s sys 0.000s
I20260807 19:49:29.749936 11196 raft_consensus.cc:697] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:29.815295 8788 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:29.796149 8778 raft_consensus.cc:2410] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:29.825055 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1135) took 1385 ms. Trace:
W20260807 19:49:29.826488 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 252661us
I20260807 19:49:29.827729 8906 rpcz_store.cc:276] 0807 19:49:28.439258 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:28.712954 (+273696us) service_pool.cc:224] Handling call
0807 19:49:28.734594 (+ 21640us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:28.734893 (+ 299us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:28.734922 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:28.735458 (+ 536us) log.cc:844] Serialized 26 byte log entry
0807 19:49:28.781291 (+ 45833us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:28.807275 (+ 25984us) log.cc:844] Serialized 14 byte log entry
0807 19:49:29.662810 (+855535us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b18003e8780. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fae05 00007f687b5fc440 00007f687b60eaf6 00007f687b61f667 00007f687b61f5e8
0807 19:49:29.784946 (+122136us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:29.785120 (+ 174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:29.785199 (+ 79us) raft_consensus.cc:1685] finished
0807 19:49:29.824836 (+ 39637us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:29.824990 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20568,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":72339,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":11320,"spinlock_wait_cycles":10149632,"thread_start_us":153128,"threads_started":2,"wal-append.queue_time_us":41271,"wal-append.run_cpu_time_us":3026,"wal-append.run_wall_time_us":1032289}
I20260807 19:49:29.831902 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.842068 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.823501 11104 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.846560 8849 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:29.849215 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1210) took 2040 ms. Trace:
I20260807 19:49:29.778831 11000 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:29.864395 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.865273 8783 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:29.820869 11254 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:29.871690 11287 log.cc:927] Time spent T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 1.111s user 0.011s sys 0.000s
I20260807 19:49:29.850963 8787 rpcz_store.cc:276] 0807 19:49:27.808554 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:28.172872 (+364318us) service_pool.cc:224] Handling call
0807 19:49:28.173215 (+ 343us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:49:28.350875 (+177660us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:28.350940 (+ 65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:49:28.377050 (+ 26110us) log.cc:844] Serialized 52 byte log entry
0807 19:49:28.563176 (+186126us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b18003132a0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a32c1 00007f687b5bfdef
0807 19:49:28.648858 (+ 85682us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:28.661934 (+ 13076us) log.cc:844] Serialized 14 byte log entry
0807 19:49:28.667981 (+ 6047us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:28.668079 (+ 98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:29.581747 (+913668us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:29.848928 (+267181us) raft_consensus.cc:1685] finished
0807 19:49:29.848958 (+ 30us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:29.849129 (+ 171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29309,"prepare.queue_time_us":15101,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":183894,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":21443,"spinlock_wait_cycles":22030464,"thread_start_us":249613,"threads_started":2,"wal-append.queue_time_us":316858}
I20260807 19:49:29.847954 11081 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.843691 11118 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.852586 11180 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:29.758147 11356 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:29.883697 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.855027 11312 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:29.873732 11314 consensus_meta.cc:287] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.776s user 0.013s sys 0.000s
I20260807 19:49:29.755620 11358 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:29.897388 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:29.841113 11060 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.879926 10895 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.915334 11232 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:29.893476 11199 consensus_meta.cc:287] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.010s user 0.003s sys 0.012s
I20260807 19:49:29.923600 11199 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.927008 8902 raft_consensus.cc:2393] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:29.889861 11180 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:29.856390 11104 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.905365 11260 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:29.915534 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:29.916435 11362 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:29.918114 10781 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:29.916431 11065 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:29.922945 11000 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.913847 8783 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:29.958231 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:29.958846 8783 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:29.894455 11202 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.926450 11199 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.931825 11180 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:29.941140 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:29.971576 9015 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:29.942292 10889 raft_consensus.cc:3055] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:29.944113 9020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 841) took 1046 ms. Trace:
I20260807 19:49:29.975028 9020 rpcz_store.cc:276] 0807 19:49:28.897303 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:28.897494 (+ 191us) service_pool.cc:224] Handling call
0807 19:49:28.897834 (+ 340us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:29.178206 (+280372us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:29.943856 (+765650us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540006b8d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:29.944048 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640066432}
I20260807 19:49:29.947468 11358 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:29.875926 11196 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
W20260807 19:49:29.952507 11262 log.cc:927] Time spent T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.157s user 0.000s sys 0.001s
I20260807 19:49:29.954523 11143 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:29.895354 11118 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.941233 11254 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.937485 11237 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:29.901329 11314 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.983850 8783 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:29.959160 11356 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:29.956981 11065 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:29.974225 10889 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:29.998883 8849 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:29.940160 11362 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:29.984724 11256 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.030898 11314 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.001346 11143 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.032476 11143 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.035019 11143 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:30.035983 11363 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.278s user 0.002s sys 0.000s
I20260807 19:49:30.036880 11234 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:30.038110 11378 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
I20260807 19:49:30.039340 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.040812 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.041469 8789 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:30.042547 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.047086 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.047672 8789 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.049041 8849 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:29.993896 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:30.008306 11358 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:30.000052 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.059127 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.046157 11330 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:30.060217 8783 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.056303 11143 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:30.071158 8789 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:30.077399 8849 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:30.080706 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.080968 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.082433 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.083511 8789 raft_consensus.cc:2410] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:30.084426 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.086863 11362 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.087213 11316 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.063283 11358 raft_consensus.cc:697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:30.089313 11118 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.089577 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:30.089881 11354 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.386s user 0.001s sys 0.000s
I20260807 19:49:30.090111 11199 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:30.138943 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.099092 11309 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.101137 9850 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.046209 11256 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.074116 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.129136 9898 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:49:30.144126 9850 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:30.076792 11143 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:30.140046 8779 raft_consensus.cc:2410] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:30.150020 8783 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:30.153587 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1052) took 1311 ms. Trace:
I20260807 19:49:30.155405 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.157981 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.160238 8779 rpcz_store.cc:276] 0807 19:49:28.842661 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:28.842808 (+ 147us) service_pool.cc:224] Handling call
0807 19:49:30.138735 (+1295927us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:30.153518 (+ 14783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9452928}
I20260807 19:49:30.160959 8783 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 2: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:30.161269 9015 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 2.
I20260807 19:49:30.162461 8776 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:30.156443 11199 raft_consensus.cc:697] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:30.163671 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.164059 8776 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:30.164520 11199 consensus_queue.cc:237] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.165046 11358 consensus_queue.cc:237] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.107295 10889 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.152647 11237 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.150853 11309 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.167701 11237 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.170125 11309 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.170966 11237 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:30.154749 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:30.162384 11323 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:30.174182 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.174618 8849 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:30.175027 8901 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:30.175679 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.152697 11362 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:30.183406 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.184702 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:30.193766 11065 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:30.195626 11202 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:30.200376 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:30.196110 11202 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.201925 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.153203 11143 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:30.207314 8783 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:30.191511 11060 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:30.237622 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.237962 9022 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:30.207497 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.183444 11362 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:30.240104 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.240732 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.240584 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.256531 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:30.208552 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.257961 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.258600 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.218305 8972 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:30.260382 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:30.260938 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.261828 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.191604 11314 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:30.262578 9017 raft_consensus.cc:2393] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:30.261134 11309 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:30.240240 11362 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:30.265874 11244 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.191831 11104 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:30.240162 11394 log.cc:927] Time spent T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
W20260807 19:49:30.243134 11357 consensus_meta.cc:287] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.574s user 0.002s sys 0.010s
W20260807 19:49:30.276527 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 58355us
I20260807 19:49:30.242368 9008 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:49:30.205660 11323 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:30.237360 11202 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.280710 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.286937 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.271864 10741 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:30.205039 11090 consensus_queue.cc:1048] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260807 19:49:30.261731 9018 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.233661 11316 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.238332 11237 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.286914 9021 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.279475 11314 raft_consensus.cc:697] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:30.201617 11180 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.278187 11082 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:30.275615 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 185431us
I20260807 19:49:30.287835 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:30.279610 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.282896 11357 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:30.327246 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.278957 11244 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.302589 8970 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:30.354516 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 52024.8us
I20260807 19:49:30.341593 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.359025 9027 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:30.353225 11072 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:30.295313 11104 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.326196 11180 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.301538 11118 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.325507 11256 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:30.342634 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 55403.3us
I20260807 19:49:30.377077 8972 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.379263 8972 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.336261 11237 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.343278 11082 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.382094 11331 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:30.373111 11072 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:30.382727 11331 raft_consensus.cc:697] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:30.384723 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:30.383297 11072 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.380980 11230 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:30.394534 9017 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.371044 11104 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.384047 10756 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:30.389180 11085 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:30.396380 11180 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.406018 9889 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260807 19:49:30.396522 11230 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:30.406821 11230 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:30.390925 11387 log.cc:927] Time spent T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.444s user 0.000s sys 0.005s
I20260807 19:49:30.391542 9018 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:30.380468 11118 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.403046 10756 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:30.425812 11400 log.cc:927] Time spent T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260807 19:49:30.426963 10977 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.426503 11181 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.406203 11180 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.383641 11331 consensus_queue.cc:237] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.442255 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:30.447427 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 50224.7us
I20260807 19:49:30.388209 11314 consensus_queue.cc:237] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.448679 10756 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.418566 11114 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260807 19:49:30.475883 11402 log.cc:927] Time spent T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.122s user 0.001s sys 0.000s
I20260807 19:49:30.474050 11085 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:30.463084 10977 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.475373 11392 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:30.517468 11403 log.cc:927] Time spent T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.114s user 0.000s sys 0.001s
W20260807 19:49:30.546725 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 74417.2us
I20260807 19:49:30.533316 11392 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:30.537621 11082 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:30.551297 11392 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:30.563099 11143 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.565336 11143 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:30.570257 8915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1158) took 1129 ms. Trace:
I20260807 19:49:30.570739 8915 rpcz_store.cc:276] 0807 19:49:29.440784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.440916 (+ 132us) service_pool.cc:224] Handling call
0807 19:49:29.441240 (+ 324us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:29.642050 (+200810us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:30.569965 (+927915us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400461ed4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:30.570179 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460036992}
I20260807 19:49:30.573312 11180 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:28.906383 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 19:49:30.548128 11333 log.cc:927] Time spent T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 1.070s user 0.002s sys 0.004s
W20260807 19:49:30.565267 11353 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.725s user 0.006s sys 0.002s
I20260807 19:49:30.546447 11237 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.581609 11104 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.619062 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 66) took 9228 ms. Trace:
I20260807 19:49:30.619599 8629 rpcz_store.cc:276] 0807 19:49:21.354951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:21.355121 (+ 170us) service_pool.cc:224] Handling call
0807 19:49:30.582971 (+9227850us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1117873,"reported_tablets":9}
I20260807 19:49:30.628968 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.596877 11392 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.623433 11309 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:30.640017 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 81954.9us
I20260807 19:49:30.626988 11118 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:30.645649 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:30.640687 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1073) took 1250 ms. Trace:
I20260807 19:49:30.640492 10782 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:30.641997 10977 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.671466 8972 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.674706 11392 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:30.675406 11086 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:30.676309 8972 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:30.659425 11085 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.666759 11417 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260807 19:49:30.692581 10364 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:30.709718 10364 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:30.710047 10364 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.718694 11019 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260807 19:49:30.668138 11202 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.681686 11086 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:49:30.720808 8788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1231) took 1487 ms (client timeout 1649 ms). Trace:
W20260807 19:49:30.765498 8788 rpcz_store.cc:269] 0807 19:49:29.233552 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.233679 (+ 127us) service_pool.cc:224] Handling call
0807 19:49:30.720542 (+1486863us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b540008a2d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:30.720705 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291880192}
I20260807 19:49:30.696651 11330 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:30.723235 11348 log.cc:927] Time spent T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 1.066s user 0.005s sys 0.001s
I20260807 19:49:30.724802 11309 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:30.726148 11362 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.693217 11244 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.728094 10977 consensus_queue.cc:1048] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:30.729897 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:30.731285 11314 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.736683 11230 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.741803 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:30.781989 9020 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.746690 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.747323 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.751138 10364 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.700588 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:30.752511 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 871) took 1210 ms. Trace:
W20260807 19:49:30.769600 8902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 866) took 1253 ms (client timeout 1613 ms). Trace:
I20260807 19:49:30.772948 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.775501 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.765254 11086 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:30.777995 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:30.780853 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 51039.4us
I20260807 19:49:30.746306 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.777155 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:30.785534 9021 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.785043 8903 rpcz_store.cc:276] 0807 19:49:29.542193 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.542319 (+ 126us) service_pool.cc:224] Handling call
0807 19:49:29.542777 (+ 458us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:29.758251 (+215474us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:29.758289 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:29.784548 (+ 26259us) log.cc:844] Serialized 26 byte log entry
0807 19:49:29.875062 (+ 90514us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:29.875131 (+ 69us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:30.383484 (+508353us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:30.752118 (+368634us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b54003606d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:30.752182 (+ 64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:30.752236 (+ 54us) raft_consensus.cc:1685] finished
0807 19:49:30.752249 (+ 13us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:30.752436 (+ 187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10694,"raft.queue_time_us":56929,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7912,"spinlock_wait_cycles":118114816,"thread_start_us":216830,"threads_started":2,"wal-append.queue_time_us":111639}
I20260807 19:49:30.786111 8901 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.785379 8911 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:30.768512 11330 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:30.787698 10364 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:30.815584 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.815953 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:30.785156 8902 rpcz_store.cc:269] 0807 19:49:29.516667 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.525010 (+ 8343us) service_pool.cc:224] Handling call
0807 19:49:30.769371 (+1244361us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54001ab3d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:30.769533 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922098944}
I20260807 19:49:30.789676 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.817173 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.817236 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.788901 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.818090 8901 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.818328 8902 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:30.818423 8906 raft_consensus.cc:2393] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:30.817204 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:30.819891 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.819211 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.820477 8910 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:30.820921 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.821025 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.815348 8913 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.821619 8901 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 1.
I20260807 19:49:30.821600 8908 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.817077 9027 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.822858 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:30.700212 10756 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.830935 8913 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:30.830855 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:30.832368 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.786269 11244 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:30.819343 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.835049 8911 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:30.836750 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1071) took 1450 ms. Trace:
I20260807 19:49:30.837317 11114 raft_consensus.cc:2764] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260807 19:49:30.833772 9893 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
W20260807 19:49:30.838214 11404 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.373s user 0.002s sys 0.000s
I20260807 19:49:30.819844 11118 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:30.845566 9020 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:30.823292 11086 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.785596 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.846273 8902 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:30.824100 9890 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:30.778424 11309 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:30.823818 11104 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:30.887013 11357 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.892896 11357 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.844892 8780 rpcz_store.cc:276] 0807 19:49:29.386517 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.450791 (+ 64274us) service_pool.cc:224] Handling call
0807 19:49:29.451111 (+ 320us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:29.451955 (+ 844us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:29.451984 (+ 29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:29.456591 (+ 4607us) log.cc:844] Serialized 26 byte log entry
0807 19:49:29.541162 (+ 84571us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:29.990371 (+449209us) log.cc:844] Serialized 14 byte log entry
0807 19:49:29.990608 (+ 237us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:29.990667 (+ 59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:30.639978 (+649311us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:30.836423 (+196445us) raft_consensus.cc:1685] finished
0807 19:49:30.836532 (+ 109us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:30.836682 (+ 150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":208951,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":1280,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":21081,"spinlock_wait_cycles":473780608,"thread_start_us":441412,"threads_started":2,"wal-append.queue_time_us":176245}
I20260807 19:49:30.893770 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.894469 8788 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:30.894811 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.895045 11357 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.895408 8780 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:30.783145 11314 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.856926 8904 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:30.857801 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.858557 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.859095 11256 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:30.862092 11358 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.902185 8910 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:30.862582 9019 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:30.862773 11181 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:30.863286 11421 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.111s user 0.000s sys 0.000s
I20260807 19:49:30.865978 10756 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:30.833984 11237 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260807 19:49:30.853741 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:30.847437 11019 tablet_bootstrap.cc:492] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Bootstrap complete.
I20260807 19:49:30.885149 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:30.932655 8790 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:30.886153 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.890452 11180 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.890908 11143 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.891274 11309 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.895328 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:30.909952 11314 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.911022 8776 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:30.918102 8780 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:30.873025 11202 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:30.923897 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:30.924616 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:30.922899 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:30.913625 11085 consensus_queue.cc:1059] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260807 19:49:30.944058 11104 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:30.851337 11255 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260807 19:49:30.896013 11411 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:30.944528 8790 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:30.950695 8788 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:30.929232 11392 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:30.955698 11019 ts_tablet_manager.cc:1403] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Time spent bootstrapping tablet: real 1.968s user 0.039s sys 0.001s
I20260807 19:49:30.926277 11358 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.949536 11118 raft_consensus.cc:476] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Not starting pre-election -- already a leader
I20260807 19:49:30.966367 11196 consensus_queue.cc:1059] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:30.967099 11217 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:30.967499 11425 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:30.971588 11114 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:30.973449 8904 raft_consensus.cc:1240] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:30.926017 11181 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.956182 10364 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:31.012553 10364 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:30.930419 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 76757.7us
I20260807 19:49:30.990549 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:30.962795 11314 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:30.993561 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:31.017786 9017 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:30.950604 11143 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.959626 10988 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:30.996006 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:31.024712 9020 raft_consensus.cc:2393] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:30.948483 11180 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.977495 11104 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.006889 11417 raft_consensus.cc:3055] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:31.007655 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.010425 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 74) took 1100 ms. Trace:
I20260807 19:49:30.996919 10782 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:30.991489 11362 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.035521 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:30.993515 11202 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:30.986356 11392 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.038795 10364 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:31.040534 8779 rpcz_store.cc:276] 0807 19:49:29.909788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:30.273029 (+363241us) service_pool.cc:224] Handling call
0807 19:49:30.273381 (+ 352us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:30.274887 (+ 1506us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:30.274919 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:30.283390 (+ 8471us) log.cc:844] Serialized 26 byte log entry
0807 19:49:30.358128 (+ 74738us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:49:30.374507 (+ 16379us) log.cc:844] Serialized 14 byte log entry
0807 19:49:30.374728 (+ 221us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:30.374791 (+ 63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:30.837643 (+462852us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:31.010079 (+172436us) raft_consensus.cc:1685] finished
0807 19:49:31.010173 (+ 94us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:31.010359 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1583,"prepare.queue_time_us":13958,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":1572,"raft.queue_time_us":2590,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":8528,"spinlock_wait_cycles":3704576,"thread_start_us":74360,"threads_started":1,"wal-append.queue_time_us":128930}
I20260807 19:49:30.983749 11411 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:31.037850 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.051492 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:31.083632 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:31.085600 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 47857.3us
I20260807 19:49:31.017813 11230 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:31.053676 9017 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:31.085594 9020 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.089411 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:31.053761 9023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37380 (request call id 103) took 1119 ms. Trace:
I20260807 19:49:31.090802 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.092381 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.031128 11107 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:31.094997 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.015045 11114 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:30.665793 8775 rpcz_store.cc:276] 0807 19:49:29.388139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.531508 (+143369us) service_pool.cc:224] Handling call
0807 19:49:29.531871 (+ 363us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:29.532270 (+ 399us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:29.532301 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:29.532906 (+ 605us) log.cc:844] Serialized 26 byte log entry
0807 19:49:29.609326 (+ 76420us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:29.831725 (+222399us) log.cc:844] Serialized 14 byte log entry
0807 19:49:29.843357 (+ 11632us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:29.858305 (+ 14948us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:30.563621 (+705316us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b18003c8fe0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784cbc56 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb
0807 19:49:30.564701 (+ 1080us) spinlock_profiling.cc:243] Waited 9.01 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:30.631370 (+ 66669us) raft_consensus.cc:1685] finished
0807 19:49:30.632873 (+ 1503us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:30.638696 (+ 5823us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2465,"prepare.queue_time_us":224647,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":5940,"raft.queue_time_us":6192,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":22895,"spinlock_wait_cycles":37620480,"thread_start_us":234773,"threads_started":2,"wal-append.queue_time_us":250560}
I20260807 19:49:31.040117 11417 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:31.040231 11104 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:31.011829 11425 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:31.104278 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:31.252580 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 240921us
I20260807 19:49:31.256734 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:31.040730 11118 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260807 19:49:31.043435 11244 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.201s
I20260807 19:49:31.094002 11255 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:31.053493 11247 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:31.264695 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:31.090687 9023 rpcz_store.cc:276] 0807 19:49:29.934587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:29.935536 (+ 949us) service_pool.cc:224] Handling call
0807 19:49:29.935943 (+ 407us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:29.936215 (+ 272us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 19:49:29.968130 (+ 31915us) log.cc:844] Serialized 14 byte log entry
0807 19:49:30.953819 (+985689us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b180036a060. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a342a 00007f687b60ee0a
0807 19:49:31.053384 (+ 99565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:31.053455 (+ 71us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:49:31.053500 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:31.053565 (+ 65us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:31.053696 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18702,"prepare.queue_time_us":31746,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":85,"raft.queue_time_us":9460,"spinlock_wait_cycles":6930688,"thread_start_us":90414,"threads_started":1}
I20260807 19:49:31.098079 11237 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:31.052409 10782 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:31.028066 11314 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.098834 11324 consensus_queue.cc:1059] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.188s
W20260807 19:49:31.294289 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 202003us
I20260807 19:49:31.314497 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.084105 11202 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.298727 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:31.043874 10988 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.051990 11086 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.242479 11107 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.344588 9015 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:31.347201 11244 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.354976 9893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33436 (request call id 124) took 1258 ms. Trace:
I20260807 19:49:31.301136 11237 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.309166 9023 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:31.290714 11255 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.314195 11306 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:31.334221 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56800.8us
I20260807 19:49:31.355538 9893 rpcz_store.cc:276] 0807 19:49:30.096494 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:30.133614 (+ 37120us) service_pool.cc:224] Handling call
0807 19:49:30.156154 (+ 22540us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:30.870454 (+714300us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:31.354458 (+484004us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400565ed4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:31.354864 (+ 406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525705472}
I20260807 19:49:31.314667 11357 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:31.360991 11433 log.cc:927] Time spent T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.317s user 0.000s sys 0.001s
W20260807 19:49:31.361582 11436 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.265s user 0.000s sys 0.000s
I20260807 19:49:31.402866 9020 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:31.405825 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.384572 11237 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.393030 11255 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.363988 11202 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.409072 11358 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.417029 11256 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:31.409454 10977 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:31.359645 11019 raft_consensus.cc:359] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 2 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.403595 11357 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:31.473837 11438 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.147s user 0.000s sys 0.000s
I20260807 19:49:31.475205 11357 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:31.475374 9890 consensus_meta.cc:287] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.555s user 0.009s sys 0.000s
I20260807 19:49:31.475946 9890 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:31.497037 11442 log.cc:927] Time spent T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.159s user 0.000s sys 0.000s
I20260807 19:49:31.498914 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1075) took 1979 ms. Trace:
I20260807 19:49:31.499329 8789 rpcz_store.cc:276] 0807 19:49:29.519287 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:30.102250 (+582963us) service_pool.cc:224] Handling call
0807 19:49:30.112652 (+ 10402us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:30.141487 (+ 28835us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:30.141524 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:30.152225 (+ 10701us) log.cc:844] Serialized 26 byte log entry
0807 19:49:30.957845 (+805620us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18003f61c0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a32c1 00007f687b5bfdef
0807 19:49:31.031578 (+ 73733us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:31.048808 (+ 17230us) log.cc:844] Serialized 14 byte log entry
0807 19:49:31.049076 (+ 268us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:31.049157 (+ 81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:31.498547 (+449390us) raft_consensus.cc:1685] finished
0807 19:49:31.498675 (+ 128us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:31.498843 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29678,"prepare.queue_time_us":17599,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":33204,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":39591,"spinlock_wait_cycles":26288896,"thread_start_us":73301,"threads_started":1,"wal-append.queue_time_us":1103131}
I20260807 19:49:31.340408 11417 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.475862 11357 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:31.467073 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 152635us
I20260807 19:49:31.501875 11417 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.502521 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:31.503410 11330 consensus_meta.cc:287] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.670s user 0.005s sys 0.003s
I20260807 19:49:31.401405 11306 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:31.499058 11437 log.cc:927] Time spent T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.135s user 0.001s sys 0.000s
W20260807 19:49:31.505052 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 43514us
I20260807 19:49:31.445060 11181 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.511941 9021 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 19:49:31.444221 11180 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.501814 11357 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.519403 9017 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260807 19:49:31.520714 11357 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.522732 11085 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:49:31.525926 11445 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260807 19:49:31.507689 11415 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:31.526819 11415 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.528899 11227 raft_consensus.cc:3055] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:31.529330 11227 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:31.507380 11447 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:31.529815 11447 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.530241 11447 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:31.534709 11425 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:31.538851 11411 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:31.540025 10988 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:31.543111 11114 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.543941 11392 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:31.545897 11306 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.546977 11019 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Initialized, Role: LEARNER
I20260807 19:49:31.547714 11019 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:31.549638 11019 ts_tablet_manager.cc:1434] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Time spent starting tablet: real 0.562s user 0.004s sys 0.004s
W20260807 19:49:31.549919 11019 ts_tablet_manager.cc:1467] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Tablet startup took 564ms
I20260807 19:49:31.505955 10977 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.552439 10977 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.538576 11309 raft_consensus.cc:3055] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:31.556833 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:31.558105 8779 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:31.573243 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:31.574417 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:31.574738 8908 raft_consensus.cc:1240] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:31.575479 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:31.568634 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:31.576866 8780 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:31.576964 9850 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:31.578029 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.468724 11358 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.586369 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:31.588626 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:31.579981 9021 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:31.571209 11309 raft_consensus.cc:740] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:31.591452 11309 consensus_queue.cc:260] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:31.593984 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.579613 10977 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:31.576089 8779 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:31.503822 11330 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.538583 11107 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.566604 8630 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:31.606532 8849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.735s user 0.011s sys 0.079s
I20260807 19:49:31.529345 11314 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.607517 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:31.608464 8849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s user 0.012s sys 0.079s
I20260807 19:49:31.613787 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:31.596941 11382 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:31.605840 8779 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:31.618300 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:31.609323 8856 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:31.610847 11082 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s user 0.013s sys 0.013s
I20260807 19:49:31.609397 8630 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 2 to 3, leader changed from 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:31.619280 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:31.596105 11447 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.621467 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:31.634335 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:31.605358 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:31.543195 11454 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260807 19:49:31.638938 8902 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:31.648957 8630 catalog_manager.cc:5697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:31.657016 8856 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.657430 11227 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.658583 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.601202 11358 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.651926 8630 catalog_manager.cc:5697] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:31.611233 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:31.596798 10977 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.669485 8630 catalog_manager.cc:5697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 2 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:31.664686 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:31.659960 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:31.637231 11082 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s user 0.014s sys 0.014s
I20260807 19:49:31.675215 8630 catalog_manager.cc:5697] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:31.680809 8630 catalog_manager.cc:5697] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:31.684222 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:31.685096 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:31.685225 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:31.685739 8915 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.638428 11181 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:31.638073 10782 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:31.658951 11227 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:31.647269 11382 raft_consensus.cc:697] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:31.623899 11310 consensus_queue.cc:1059] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260807 19:49:31.698127 8856 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.693792 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:31.699640 8740 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.708s (SENT) [suppressed 53 similar messages]
I20260807 19:49:31.696091 8630 catalog_manager.cc:5697] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:31.688265 11180 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:31.723084 8630 catalog_manager.cc:5697] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 2 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:31.727003 8915 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:31.704914 11454 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:31.741915 8915 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:31.717159 11357 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:31.706930 11114 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:31.695348 11456 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:31.690507 11449 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:31.705269 11256 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260807 19:49:31.705075 11247 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260807 19:49:31.763572 8915 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:31.766223 8915 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260807 19:49:31.746815 11454 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:31.770747 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 72686.4us
I20260807 19:49:31.774542 8856 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.763427 8775 raft_consensus.cc:2410] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:31.755362 11114 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:31.791159 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:31.751994 11461 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.807107 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:31.754469 11449 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:31.796587 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:31.771864 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 170049us
I20260807 19:49:31.767755 11230 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:31.777073 11454 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:31.822228 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:31.773380 11453 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:31.781206 11357 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:31.784276 11256 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:31.794775 11114 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:31.843431 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 135268us
I20260807 19:49:31.853838 8849 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:31.835716 11230 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.855496 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:31.856556 11306 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:31.857751 11306 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:31.858148 11306 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:31.858491 11306 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:31.870004 8972 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:31.828088 11330 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.816856 11449 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:31.818086 11464 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:31.835723 11446 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:31.839654 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:31.920915 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 49349.5us
W20260807 19:49:31.930619 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60687.7us
I20260807 19:49:31.931284 11465 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:31.938077 8972 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:31.838460 11180 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.084s
W20260807 19:49:31.962245 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 262595us
I20260807 19:49:31.916126 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:31.793787 11382 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:31.867791 11357 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.939203 11465 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:31.920804 11055 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:31.871556 11256 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:31.945039 11358 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:31.821964 11461 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.024195 11461 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.026006 8779 raft_consensus.cc:1275] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260807 19:49:32.021098 8903 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:32.026427 8779 pending_rounds.cc:85] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:32.026705 8779 pending_rounds.cc:107] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:32.026953 8779 raft_consensus.cc:2889] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:32.031309 11465 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:32.037856 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 226184us
I20260807 19:49:31.962251 11447 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.040673 11456 raft_consensus.cc:697] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:32.054683 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:31.975720 11453 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.055400 8910 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:32.020092 11221 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:31.941088 11446 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:31.989284 11310 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260807 19:49:32.015551 10977 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:32.064669 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:32.079279 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:32.042279 11055 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.092850 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 155648us
I20260807 19:49:32.078276 11221 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:32.106557 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:32.085305 11256 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.087271 11491 raft_consensus.cc:3055] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:32.101652 11490 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.065546 11453 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.109058 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 287198us
W20260807 19:49:32.146955 11237 consensus_meta.cc:287] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.724s user 0.005s sys 0.006s
I20260807 19:49:32.203143 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.206566 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.220548 8787 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:32.221606 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.222752 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.206866 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.225288 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.223814 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:31.554230 11019 ts_tablet_manager.cc:1469] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Trace:
0807 19:49:24.846635 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.848499 (+ 1864us) service_pool.cc:224] Handling call
0807 19:49:25.109158 (+260659us) ts_tablet_manager.cc:934] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Initiating tablet copy from peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
0807 19:49:26.580108 (+1470950us) tablet_copy_client.cc:368] Tablet copy session begun
0807 19:49:27.204785 (+624677us) tablet_metadata.cc:665] Metadata flushed
0807 19:49:27.204955 (+ 170us) tablet_copy_client.cc:497] Wrote new tablet metadata
0807 19:49:27.730875 (+525920us) tablet_copy_client.cc:418] Wrote new consensus metadata
0807 19:49:27.765433 (+ 34558us) tablet_replica.cc:206] Creating consensus instance
0807 19:49:27.800058 (+ 34625us) ts_tablet_manager.cc:989] Replica registered.
0807 19:49:27.806343 (+ 6285us) inbound_call.cc:177] Queueing success response
0807 19:49:28.980749 (+1174406us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0807 19:49:30.988121 (+2007372us) ts_tablet_manager.cc:1435] Starting tablet replica
0807 19:49:31.356943 (+368822us) tablet_replica.cc:261] Starting instrumentation
0807 19:49:31.357608 (+ 665us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":35620,"mutex_wait_us":7286,"raft.queue_time_us":4971,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":27254,"tablet-copy.queue_time_us":223624,"tablet-download-pool.queue_time_us":157,"tablet-download-pool.run_cpu_time_us":1494,"tablet-download-pool.run_wall_time_us":304826,"thread_start_us":217388,"threads_started":2}
I20260807 19:49:32.205821 8790 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:32.216841 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:32.227355 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.228813 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.229907 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:32.230903 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.231462 8790 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:32.232566 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.233733 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.216547 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 844) took 1995 ms. Trace:
I20260807 19:49:32.235610 9019 rpcz_store.cc:276] 0807 19:49:30.207426 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:30.237518 (+ 30092us) service_pool.cc:224] Handling call
0807 19:49:30.714233 (+476715us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c684c 00007f68784c640b 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:30.714499 (+ 266us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:30.923378 (+208879us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:32.172774 (+1249396us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b540041e6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:32.203082 (+ 30308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316682368}
I20260807 19:49:32.194856 8968 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:32.217044 11491 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:32.093703 11314 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:32.145923 11417 consensus_meta.cc:287] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.643s user 0.010s sys 0.000s
I20260807 19:49:32.205935 11490 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.240332 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 45592.2us
I20260807 19:49:32.256299 8968 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:32.098927 10977 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.241676 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:32.111353 11477 log.cc:927] Time spent T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.258s user 0.000s sys 0.001s
I20260807 19:49:32.248268 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:32.176692 11244 consensus_meta.cc:287] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.829s user 0.013s sys 0.002s
I20260807 19:49:32.251159 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.260596 9027 raft_consensus.cc:3055] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:32.195079 11464 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:32.265969 9027 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:32.264918 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.262095 8968 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:32.108682 11486 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.266350 9015 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:32.268792 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.239329 11256 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:32.269794 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.218710 11461 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:32.204731 11497 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.315543 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:32.306962 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:32.307358 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.218904 11358 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.307979 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.267083 9027 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.319335 9027 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:32.308763 8787 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:32.320103 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.195274 11455 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:32.326242 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.327442 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.245008 11468 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:32.270222 11357 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.317615 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.303436 11244 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:32.304129 11444 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:32.245240 11237 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.102833 11456 consensus_queue.cc:237] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.328583 11358 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.260445 11314 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.243500 11465 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.303504 10977 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.390408 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.403692 8913 raft_consensus.cc:1240] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:32.359937 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:32.440845 8906 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
W20260807 19:49:32.467170 8737 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.660s user 0.007s sys 0.016s
W20260807 19:49:32.476403 8737 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s user 0.007s sys 0.017s
I20260807 19:49:32.475529 8908 raft_consensus.cc:2393] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:32.344617 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.407563 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.477013 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.482353 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:32.395500 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:32.480808 8911 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260807 19:49:32.476831 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 669929us
W20260807 19:49:32.484392 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:32.486572 8778 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:32.487458 8911 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:32.481635 8779 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:32.489423 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.489123 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.490973 8779 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:32.486312 8913 raft_consensus.cc:1275] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260807 19:49:32.493635 8779 raft_consensus.cc:1275] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:32.329555 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.317324 11417 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:32.379093 11494 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
W20260807 19:49:32.324232 10988 consensus_queue.cc:1175] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 19:49:32.527570 9023 raft_consensus.cc:1275] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:32.517549 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:32.379208 11500 log.cc:927] Time spent T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.200s user 0.000s sys 0.000s
I20260807 19:49:32.314716 11464 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.521587 11237 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.521984 11465 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.359254 11526 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.522679 8790 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:32.334733 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.474560 11444 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:32.318331 11256 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.378211 11055 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.360137 11466 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:32.531451 11417 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:32.324797 11461 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.246156 11221 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.378568 11512 log.cc:927] Time spent T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.061s user 0.001s sys 0.000s
I20260807 19:49:32.538985 9027 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:32.472185 11447 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:32.581715 11447 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.340137 11409 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.188s
I20260807 19:49:32.582242 11447 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:32.328671 11114 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.524820 11499 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:32.586272 11499 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:32.586736 11499 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:32.467159 11446 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:32.523978 11505 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:32.539780 11306 consensus_meta.cc:287] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.681s user 0.008s sys 0.007s
W20260807 19:49:32.560115 11104 consensus_meta.cc:287] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 1.282s user 0.013s sys 0.001s
I20260807 19:49:32.526067 8779 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:32.600734 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.601435 8779 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
W20260807 19:49:32.565564 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:32.602908 8968 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:32.540603 11244 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:32.325053 11497 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.605283 11446 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:32.605922 11446 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:32.571556 11490 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.550248 11526 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.333050 11486 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.578616 11055 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:32.579308 11453 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.556602 11444 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:32.554888 11237 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:32.567111 11466 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:32.598634 11516 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.606642 8780 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
W20260807 19:49:32.605439 11492 log.cc:927] Time spent T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.101s user 0.000s sys 0.000s
I20260807 19:49:32.610481 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:32.606417 11446 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:32.408089 11491 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.610960 8910 consensus_meta.cc:287] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.551s user 0.010s sys 0.001s
I20260807 19:49:32.582000 11086 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.623941 8910 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:32.627530 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.629519 8783 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:32.626192 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1168) took 1005 ms. Trace:
I20260807 19:49:32.637979 8910 rpcz_store.cc:276] 0807 19:49:31.633866 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.638647 (+ 4781us) service_pool.cc:224] Handling call
0807 19:49:32.054500 (+415853us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b58001e4b98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:32.626119 (+571619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9294720}
I20260807 19:49:32.636049 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:32.642622 11518 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.201s user 0.000s sys 0.001s
I20260807 19:49:32.643342 11392 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:32.643831 11392 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:32.624747 11454 consensus_meta.cc:287] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.782s user 0.012s sys 0.001s
I20260807 19:49:32.620592 11466 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:32.598726 10977 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:32.651443 8972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s user 0.002s sys 0.005s
W20260807 19:49:32.652660 8972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.713s user 0.003s sys 0.005s
W20260807 19:49:32.653324 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 715313us
I20260807 19:49:32.632380 11491 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:32.604571 11505 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.644222 11392 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:32.653478 11255 consensus_meta.cc:287] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 1.211s user 0.015s sys 0.000s
I20260807 19:49:32.640571 11490 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.607005 11306 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.667083 11411 consensus_queue.cc:1059] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:32.609512 11104 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.625558 11237 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.667726 11107 raft_consensus.cc:3055] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:32.609751 11244 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.681674 11528 log.cc:927] Time spent T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.151s user 0.001s sys 0.000s
W20260807 19:49:32.689600 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:32.660144 10977 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.678071 11490 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.717546 8906 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
W20260807 19:49:32.718039 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 52917.9us
I20260807 19:49:32.724114 8972 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:32.665361 11358 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.638084 11516 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.675407 11255 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.721442 11086 consensus_queue.cc:1059] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.022s
I20260807 19:49:32.728106 11065 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:32.745765 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:32.641185 11417 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.727236 8783 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:32.688865 11107 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:32.726613 8972 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:32.655797 11454 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.749361 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1279) took 1190 ms. Trace:
I20260807 19:49:32.756049 8972 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:32.688553 11409 raft_consensus.cc:3055] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:32.760610 11526 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.765136 8788 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:32.755386 11065 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:32.768702 11065 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:32.772461 11065 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.721634 10977 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.794533 11463 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 1.021s user 0.001s sys 0.001s
I20260807 19:49:32.774747 11065 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.795512 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:32.808951 11244 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:32.812250 11507 log.cc:927] Time spent T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.121s user 0.000s sys 0.000s
I20260807 19:49:32.774879 11114 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.812180 11104 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.792512 11411 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:32.768025 11497 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.792858 11461 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:32.792268 11196 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:32.804030 11449 consensus_meta.cc:287] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.882s user 0.004s sys 0.009s
I20260807 19:49:32.795262 11505 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.829833 9022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1093) took 1816 ms. Trace:
I20260807 19:49:32.753623 8776 rpcz_store.cc:276] 0807 19:49:31.558779 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.570395 (+ 11616us) service_pool.cc:224] Handling call
0807 19:49:32.745537 (+1175142us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:32.749286 (+ 3749us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4228352}
I20260807 19:49:32.830300 9022 rpcz_store.cc:276] 0807 19:49:31.013498 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.013747 (+ 249us) service_pool.cc:224] Handling call
0807 19:49:31.014112 (+ 365us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:31.265539 (+251427us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:31.265574 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:31.265641 (+ 67us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:31.265673 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:32.829547 (+1563874us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540044a054. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:32.829609 (+ 62us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:32.829763 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577191808}
I20260807 19:49:32.816047 11454 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.763885 11525 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:32.810877 11464 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:32.853174 11521 log.cc:927] Time spent T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.192s user 0.000s sys 0.001s
I20260807 19:49:32.831341 11449 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.855554 8779 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:32.859091 11449 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.860808 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1277) took 1318 ms. Trace:
I20260807 19:49:32.861316 8775 rpcz_store.cc:276] 0807 19:49:31.542363 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.554228 (+ 11865us) service_pool.cc:224] Handling call
0807 19:49:32.860174 (+1305946us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b54000974d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:32.860714 (+ 540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122090368}
I20260807 19:49:32.862409 11256 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:32.845008 11227 consensus_meta.cc:287] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.149s user 0.011s sys 0.004s
I20260807 19:49:32.844491 11486 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:32.764573 11409 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:32.870740 11490 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:32.769250 11107 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:32.886658 8740 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.032s sys 0.056s
I20260807 19:49:32.832540 11196 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:32.893895 8740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s user 0.032s sys 0.057s
I20260807 19:49:32.867347 11306 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.871634 11237 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:32.917869 11444 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.918933 11505 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.919435 9020 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 19:49:32.919606 11505 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.920673 11444 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:32.894815 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 646598us
I20260807 19:49:32.924921 11255 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.930338 11447 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.931306 11446 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.933277 11143 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.935361 8783 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:32.868140 11525 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.942184 11525 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:32.922729 11466 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.950162 8783 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260807 19:49:32.954142 8856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s user 0.001s sys 0.045s
W20260807 19:49:32.954567 8856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s user 0.001s sys 0.045s
I20260807 19:49:32.951783 11466 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.955855 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:32.956157 8736 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.631s user 0.000s sys 0.043s
W20260807 19:49:32.957216 8736 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.643s user 0.001s sys 0.043s
I20260807 19:49:32.936158 11392 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.960995 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:32.961913 8910 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:32.953163 11409 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:32.964689 11196 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.965929 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.966688 10889 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:32.970206 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.954938 11505 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.961905 11447 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:32.948516 11499 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:32.983644 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:32.984432 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60111.4us
I20260807 19:49:32.985388 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:32.988015 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.988989 9027 raft_consensus.cc:2410] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:49:32.960969 11533 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:32.995663 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:32.996687 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:32.964762 11409 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:32.980859 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.005131 8856 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:33.008749 11536 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:33.009845 11230 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:32.997442 9022 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
W20260807 19:49:33.015931 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 259945us
W20260807 19:49:33.019541 8915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s user 0.018s sys 0.026s
I20260807 19:49:33.017324 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:32.995667 11118 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:49:33.031909 8915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.985s user 0.018s sys 0.026s
I20260807 19:49:33.034148 8972 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:32.994571 11143 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:33.036716 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:33.038687 8854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.822s user 0.002s sys 0.029s
I20260807 19:49:33.041356 8915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1173) took 1406 ms. Trace:
I20260807 19:49:33.041810 8915 rpcz_store.cc:276] 0807 19:49:31.635016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.776647 (+141631us) service_pool.cc:224] Handling call
0807 19:49:31.777015 (+ 368us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:31.777236 (+ 221us) raft_consensus.cc:1534] Early marking committed up to index 2
0807 19:49:32.047010 (+269774us) log.cc:844] Serialized 14 byte log entry
0807 19:49:33.041011 (+994001us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:33.041066 (+ 55us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:33.041106 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.041172 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.041296 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":266942,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":1229913,"threads_started":2}
W20260807 19:49:33.041396 8854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s user 0.003s sys 0.029s
I20260807 19:49:33.042685 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:33.056095 10364 consensus_queue.cc:1059] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 19:49:33.055200 8970 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s user 0.010s sys 0.012s
W20260807 19:49:33.058635 8970 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s user 0.010s sys 0.012s
I20260807 19:49:33.033874 11118 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:33.035631 9893 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
W20260807 19:49:33.069057 8789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.024s user 0.009s sys 0.018s
W20260807 19:49:33.071655 8789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.026s user 0.009s sys 0.018s
W20260807 19:49:33.066742 9850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s user 0.007s sys 0.016s
I20260807 19:49:33.035693 11237 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.038599 11230 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:32.995560 11533 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.058476 11486 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:33.057031 11392 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:33.073964 9850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s user 0.007s sys 0.016s
W20260807 19:49:33.090390 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 775144us
I20260807 19:49:33.070619 11446 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:33.065374 10364 raft_consensus.cc:3055] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:33.093891 10364 raft_consensus.cc:740] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:33.094758 10364 consensus_queue.cc:260] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.062502 11181 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.079522 11464 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:33.069901 11118 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:33.090708 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 48108.3us
I20260807 19:49:33.038450 11545 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:33.054873 11143 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.066011 11466 consensus_queue.cc:1059] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260807 19:49:33.103955 10364 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:33.104496 11392 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260807 19:49:33.078708 11415 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:33.108978 8737 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.010s sys 0.020s
W20260807 19:49:33.148972 8737 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s user 0.011s sys 0.020s
I20260807 19:49:33.096199 11256 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.069993 11515 consensus_queue.cc:1048] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.054s
I20260807 19:49:33.131328 11499 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.086757 11454 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.073071 11444 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260807 19:49:33.143296 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.143524 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.143798 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.144106 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.165647 8913 raft_consensus.cc:2368] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:33.144513 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.087572 11486 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:33.150707 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.166148 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.096740 11449 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:33.086594 11227 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260807 19:49:33.096632 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:33.170321 9023 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260807 19:49:33.125265 11464 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.163199 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.163753 11143 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:33.131767 11545 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.156575 11306 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.171985 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.096217 11358 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
W20260807 19:49:33.200466 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.170830 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.202342 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.204083 9022 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:33.164499 9897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:33.207739 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:33.176239 8904 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.207s user 0.016s sys 0.023s
I20260807 19:49:33.169791 11486 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:33.214054 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.158886 11491 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.160863 11447 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.216361 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:33.212870 8904 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.252s user 0.017s sys 0.023s
I20260807 19:49:33.243678 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.246887 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.250020 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.249729 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.247695 8903 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
W20260807 19:49:33.255278 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.207682 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:33.263125 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.193873 11449 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.280268 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.281544 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.185010 11454 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.287578 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.288118 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.288210 8903 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:33.207655 11306 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.292270 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.245100 11505 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:33.311452 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.241923 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.399369 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.399272 8915 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:33.405627 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.320889 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.407696 9022 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:33.408013 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1289) took 1596 ms. Trace:
I20260807 19:49:33.408295 8903 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:33.287578 8776 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:33.282271 11456 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.397477 11533 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.377959 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.434092 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.290118 11077 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.408650 8789 rpcz_store.cc:276] 0807 19:49:31.811745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.811910 (+ 165us) service_pool.cc:224] Handling call
0807 19:49:31.812272 (+ 362us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:31.812549 (+ 277us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:31.812910 (+ 361us) log.cc:844] Serialized 14 byte log entry
0807 19:49:33.407270 (+1594360us) log.cc:844] Serialized 14 byte log entry
0807 19:49:33.407643 (+ 373us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:33.407699 (+ 56us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:33.407741 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.407809 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.407951 (+ 142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1011,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":64744,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8483,"thread_start_us":1592584,"threads_started":3,"wal-append.queue_time_us":297283}
I20260807 19:49:33.453799 8903 raft_consensus.cc:2393] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:33.389972 11244 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.457695 11244 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.455853 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:33.460748 11547 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.141s user 0.001s sys 0.000s
I20260807 19:49:33.462182 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.462641 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.463245 8903 raft_consensus.cc:2393] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:33.463721 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.464272 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.464314 8913 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:33.465641 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.465816 11306 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.468813 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:33.469488 8903 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:33.463218 8911 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:49:33.470717 10889 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.504s user 0.008s sys 0.004s
I20260807 19:49:33.471330 9898 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:33.471854 11457 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.471977 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.472486 11545 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:33.472680 11457 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.242789 11447 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.412948 11505 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.476444 8908 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:33.484934 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.485648 9017 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:33.477545 8913 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:33.230270 11491 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.482733 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.483542 8903 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:33.386711 11554 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.406129 11256 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.397861 11550 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.386111 11556 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.550072 11491 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.551250 11464 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:33.483955 11456 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.534647 11554 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.531368 11556 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:33.598114 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 435943us
I20260807 19:49:33.603611 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1302) took 1016 ms. Trace:
I20260807 19:49:33.462144 8789 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:33.604916 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.606516 8776 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:33.607281 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.608446 8789 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:33.604387 8788 rpcz_store.cc:276] 0807 19:49:32.587426 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.971735 (+384309us) service_pool.cc:224] Handling call
0807 19:49:32.972281 (+ 546us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:49:32.973902 (+ 1621us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:32.973932 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:49:32.993608 (+ 19676us) log.cc:844] Serialized 525 byte log entry
0807 19:49:33.406945 (+413337us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:33.406989 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.407060 (+ 71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:33.602866 (+195806us) raft_consensus.cc:1685] finished
0807 19:49:33.602949 (+ 83us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.603343 (+ 394us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"thread_start_us":376226,"threads_started":1,"wal-append.queue_time_us":511348}
I20260807 19:49:33.612840 8788 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:33.632380 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1169) took 1997 ms. Trace:
I20260807 19:49:33.633062 8901 rpcz_store.cc:276] 0807 19:49:31.634171 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.745319 (+111148us) service_pool.cc:224] Handling call
0807 19:49:31.745731 (+ 412us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:31.772786 (+ 27055us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:31.772822 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:33.010073 (+1237251us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b2c0007bb78. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6879effc71 00007f6879f0f5d2 00007f687b63fde1 00007f687b60481a 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22
0807 19:49:33.030216 (+ 20143us) log.cc:844] Serialized 26 byte log entry
0807 19:49:33.380284 (+350068us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:33.380332 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.380430 (+ 98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:33.631465 (+251035us) raft_consensus.cc:1685] finished
0807 19:49:33.631515 (+ 50us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.632007 (+ 492us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26634,"raft.queue_time_us":29454,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":13742,"spinlock_wait_cycles":21600640,"thread_start_us":319354,"threads_started":1,"wal-append.queue_time_us":479625}
I20260807 19:49:33.636511 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 892) took 2030 ms. Trace:
W20260807 19:49:33.636898 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 394313us
W20260807 19:49:33.639041 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.637265 8904 rpcz_store.cc:276] 0807 19:49:31.606639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.611573 (+ 4934us) service_pool.cc:224] Handling call
0807 19:49:31.664059 (+ 52486us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:31.759035 (+ 94976us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:31.762627 (+ 3592us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:31.843082 (+ 80455us) log.cc:844] Serialized 26 byte log entry
0807 19:49:33.255095 (+1412013us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:33.282279 (+ 27184us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.289838 (+ 7559us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:33.635842 (+346004us) raft_consensus.cc:1685] finished
0807 19:49:33.635895 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.636269 (+ 374us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53521,"raft.queue_time_us":64870,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":16998,"thread_start_us":1290522,"threads_started":1,"wal-append.queue_time_us":1542488}
I20260807 19:49:33.481161 11077 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.661518 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:33.663616 9889 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33452 (request call id 178) took 1858 ms (client timeout 2128 ms). Trace:
I20260807 19:49:33.553570 11550 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.678685 11554 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.679427 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1268) took 2802 ms. Trace:
I20260807 19:49:33.681181 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.681522 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1301) took 1109 ms. Trace:
I20260807 19:49:33.682003 8779 rpcz_store.cc:276] 0807 19:49:32.571916 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.610199 (+ 38283us) service_pool.cc:224] Handling call
0807 19:49:33.680953 (+1070754us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000979d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:33.681252 (+ 299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537149312}
I20260807 19:49:33.680157 8778 rpcz_store.cc:276] 0807 19:49:30.876930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:30.877097 (+ 167us) service_pool.cc:224] Handling call
0807 19:49:31.544163 (+667066us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b58001e0398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:31.544449 (+ 286us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:32.487036 (+942587us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.678668 (+1191632us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400262d54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:33.679151 (+ 483us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2196624512}
W20260807 19:49:33.664256 9889 rpcz_store.cc:269] 0807 19:49:31.805058 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:31.830238 (+ 25180us) service_pool.cc:224] Handling call
0807 19:49:33.660900 (+1830662us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b5800192f98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c689e 000000000038d209 00007f6880585940 00007f688064ded4 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:33.663448 (+ 2548us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113268608}
I20260807 19:49:33.528697 11256 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:33.692414 8780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1095) took 1477 ms (client timeout 1737 ms). Trace:
W20260807 19:49:33.689670 11525 consensus_meta.cc:287] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.731s user 0.015s sys 0.000s
I20260807 19:49:33.681356 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:33.694341 8780 rpcz_store.cc:269] 0807 19:49:32.207178 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.225027 (+ 17849us) service_pool.cc:224] Handling call
0807 19:49:33.684114 (+1459087us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54000afd54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:33.684389 (+ 275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875668352}
I20260807 19:49:33.696560 11077 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:33.699893 11449 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:33.700131 11586 log.cc:927] Time spent T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.129s user 0.000s sys 0.001s
I20260807 19:49:33.700453 11505 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:33.701052 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 271429us
I20260807 19:49:33.702481 11456 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.710799 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.711084 11454 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.711087 11244 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:33.715934 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.720932 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.728401 9015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1114) took 1474 ms. Trace:
I20260807 19:49:33.728999 9015 rpcz_store.cc:276] 0807 19:49:32.253835 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.254092 (+ 257us) service_pool.cc:224] Handling call
0807 19:49:32.254372 (+ 280us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:32.310080 (+ 55708us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.727696 (+1417616us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b540043cbd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:33.728101 (+ 405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15403776}
I20260807 19:49:33.732646 11256 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.733042 11447 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.734805 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.738629 9021 raft_consensus.cc:2393] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:33.725260 11525 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:33.740875 9021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57870 (request call id 1121) took 1412 ms (client timeout 1753 ms). Trace:
W20260807 19:49:33.741456 9021 rpcz_store.cc:269] 0807 19:49:32.328905 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.329109 (+ 204us) service_pool.cc:224] Handling call
0807 19:49:33.733681 (+1404572us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:33.740779 (+ 7098us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4877056}
I20260807 19:49:33.740897 11454 consensus_queue.cc:1059] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260807 19:49:33.746462 11454 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260807 19:49:33.747900 9838 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.765986 8778 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:33.770759 9022 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:33.772714 8779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:60338
I20260807 19:49:33.772608 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.775341 8779 raft_consensus.cc:606] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:33.775838 8779 raft_consensus.cc:614] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 19:49:33.778317 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 0ba47e79caa54c409c53304a66ce0160 (from 8e9ad2b5e2a94d4faff90af735c6d792 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:33.784817 11525 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.777920 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.789222 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.790087 8788 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:33.791496 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:33.792667 8789 raft_consensus.cc:2393] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:33.793869 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:33.794755 8779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:60338
I20260807 19:49:33.795420 8779 raft_consensus.cc:606] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:33.795830 8779 raft_consensus.cc:614] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:33.796123 9889 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 19:49:33.796834 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.797479 9891 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:33.798707 11314 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:33.800124 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 2d69c395b87b43c0bf7c3d1efb63e699 (from 8e9ad2b5e2a94d4faff90af735c6d792 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:33.806296 11550 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:33.815007 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:33.816774 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.820977 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:33.827562 8779 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:33.828219 11055 raft_consensus.cc:3055] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:33.828807 11055 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:33.824371 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:33.836323 10977 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.838455 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.830340 11055 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:33.840394 11118 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.642s user 0.020s sys 0.006s
I20260807 19:49:33.841728 9889 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEARNER]: Advancing to term 3
I20260807 19:49:33.854842 11446 raft_consensus.cc:3055] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:33.852466 8788 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:33.841707 8740 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:33.854944 8789 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:33.861403 11588 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.108s user 0.001s sys 0.000s
W20260807 19:49:33.861940 8787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1106) took 1651 ms (client timeout 1714 ms). Trace:
W20260807 19:49:33.879266 8787 rpcz_store.cc:269] 0807 19:49:32.210024 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.223595 (+ 13571us) service_pool.cc:224] Handling call
0807 19:49:33.861549 (+1637954us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b540009ef54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:33.861779 (+ 230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189822208}
I20260807 19:49:33.862965 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1111) took 1651 ms. Trace:
I20260807 19:49:33.863600 11055 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:33.864143 11409 consensus_meta.cc:287] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.858s user 0.009s sys 0.014s
W20260807 19:49:33.863044 11486 consensus_meta.cc:287] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.619s user 0.012s sys 0.004s
I20260807 19:49:33.865726 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.866384 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:33.857527 11446 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:33.864734 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:33.869583 11516 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.870976 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:33.871776 11524 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:33.884670 11524 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.886756 11457 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:33.887116 11524 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:33.886924 11446 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.880455 8790 rpcz_store.cc:276] 0807 19:49:32.211332 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.233515 (+ 22183us) service_pool.cc:224] Handling call
0807 19:49:33.862622 (+1629107us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b5400096ad4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:33.862800 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":659979008}
I20260807 19:49:33.882726 10977 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.889912 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.860131 11118 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.891943 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.892246 11082 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.891762 9023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:33.893750 9023 raft_consensus.cc:606] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:33.883463 9020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1129) took 1501 ms. Trace:
I20260807 19:49:33.895066 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:33.895853 11486 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.894855 8968 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:33.894409 9020 rpcz_store.cc:276] 0807 19:49:32.380442 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.517527 (+137085us) service_pool.cc:224] Handling call
0807 19:49:32.517859 (+ 332us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:32.919959 (+402100us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.878560 (+958601us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54002945d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:33.881969 (+ 3409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922809600}
I20260807 19:49:33.898568 8790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f72bb6e69514b33881da72833a29378"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:60338
I20260807 19:49:33.899085 11486 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.899276 8790 raft_consensus.cc:606] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:33.899722 8790 raft_consensus.cc:614] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:33.899940 11409 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.900954 11409 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:33.901870 11331 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:33.902323 11314 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.903117 11314 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:33.903586 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 62047.9us
I20260807 19:49:33.904824 8740 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:33.905627 11314 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.905421 11118 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:33.906220 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 7f72bb6e69514b33881da72833a29378 (from 8e9ad2b5e2a94d4faff90af735c6d792 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:33.910646 10977 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:33.911120 10977 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:33.911499 10977 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:33.912029 10988 consensus_queue.cc:1059] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:33.911533 8790 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95f757dd3f63420fad091c3802390158"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:60338
I20260807 19:49:33.913403 10988 raft_consensus.cc:3055] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:33.913507 8790 raft_consensus.cc:606] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:33.914332 8790 raft_consensus.cc:614] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:33.914136 10988 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:33.917225 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 95f757dd3f63420fad091c3802390158 (from 8e9ad2b5e2a94d4faff90af735c6d792 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:33.918447 10988 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.915336 11331 consensus_queue.cc:1059] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:33.919939 8787 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:60338
I20260807 19:49:33.920401 10988 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:33.920733 8787 raft_consensus.cc:606] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:33.921378 11331 raft_consensus.cc:3055] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:33.921770 11331 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:33.922487 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 19:49:33.922906 11331 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.924533 11331 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:33.925366 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:33.931393 11524 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:33.935791 11486 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:33.936484 11486 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:33.937719 11587 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.174s user 0.003s sys 0.000s
I20260807 19:49:33.938786 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:33.940596 11118 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:33.939657 10977 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.937775 11486 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.946125 11455 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.946776 11455 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:33.948418 11556 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.949051 11556 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:33.949728 11556 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.950692 11455 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:33.947021 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:33.951963 11556 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:33.952446 11455 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:33.956666 11455 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:33.955943 8968 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:33.954833 11522 consensus_queue.cc:1059] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:33.962338 11522 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:33.955371 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.954691 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.963783 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1305) took 1212 ms. Trace:
I20260807 19:49:33.953073 11415 consensus_queue.cc:1059] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:33.931305 9889 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:33.961431 11455 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:33.962651 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:33.967859 8789 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:33.968588 11556 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:33.971434 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1297) took 1432 ms. Trace:
I20260807 19:49:33.973388 8783 rpcz_store.cc:276] 0807 19:49:32.539362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.958410 (+419048us) service_pool.cc:224] Handling call
0807 19:49:33.864530 (+906120us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:33.864837 (+ 307us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:33.864860 (+ 23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:33.864927 (+ 67us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:33.864956 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.865031 (+ 75us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.971076 (+106045us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b5400096ad0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:33.971222 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995057536}
I20260807 19:49:33.964257 8776 rpcz_store.cc:276] 0807 19:49:32.751850 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:33.607610 (+855760us) service_pool.cc:224] Handling call
0807 19:49:33.608306 (+ 696us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:33.614049 (+ 5743us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:33.614085 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:33.615017 (+ 932us) log.cc:844] Serialized 26 byte log entry
0807 19:49:33.693954 (+ 78937us) raft_consensus.cc:1645] Marking committed up to 1
0807 19:49:33.695931 (+ 1977us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.696018 (+ 87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:33.936797 (+240779us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:33.963410 (+ 26613us) raft_consensus.cc:1685] finished
0807 19:49:33.963432 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:33.963702 (+ 270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133028,"raft.queue_time_us":29636,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":133409,"spinlock_wait_cycles":7918336,"thread_start_us":59479,"threads_started":1,"wal-append.queue_time_us":124248}
I20260807 19:49:33.965525 11522 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:33.965323 11444 raft_consensus.cc:3055] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:33.977414 11444 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:33.976927 11522 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.978561 11522 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:33.979103 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:33.980680 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:33.981678 9008 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:33.983184 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.980912 11314 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:33.994146 11314 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:33.996363 11107 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:33.996905 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:33.997567 8902 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:33.983196 9893 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33434 (request call id 113) took 1380 ms. Trace:
I20260807 19:49:33.998632 9893 rpcz_store.cc:276] 0807 19:49:32.603016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.607119 (+ 4103us) service_pool.cc:224] Handling call
0807 19:49:32.611421 (+ 4302us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:33.075602 (+464181us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.976310 (+900708us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54005affd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:33.983013 (+ 6703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":917613440}
I20260807 19:49:33.976872 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.003485 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1292) took 1468 ms. Trace:
I20260807 19:49:34.003940 8775 rpcz_store.cc:276] 0807 19:49:32.535122 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:32.862466 (+327344us) service_pool.cc:224] Handling call
0807 19:49:33.863525 (+1001059us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:33.863944 (+ 419us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:33.863982 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:33.864057 (+ 75us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:33.864116 (+ 59us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:33.864200 (+ 84us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:34.003144 (+138944us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b540009ef50. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:34.003305 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2301109120}
I20260807 19:49:34.005607 8779 raft_consensus.cc:2410] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260807 19:49:34.006059 11118 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.006552 11118 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.007304 8737 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.009578 11107 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.010427 11446 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:33.988626 8901 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:33.988991 11556 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.012688 11556 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.018247 11466 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:33.978596 11444 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.009452 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:33.953174 10977 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:34.024297 9898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33436 (request call id 125) took 3833 ms (client timeout 2327 ms). Trace:
I20260807 19:49:34.024788 8970 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:34.027527 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.029609 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.030233 10756 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:34.031677 9838 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:34.032114 11309 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:34.060997 9898 rpcz_store.cc:269] 0807 19:49:30.191245 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:30.191930 (+ 685us) service_pool.cc:224] Handling call
0807 19:49:34.023018 (+3831088us) spinlock_profiling.cc:243] Waited 3.27 s on lock 0x7b5400195354. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:34.024072 (+ 1054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7520977792}
W20260807 19:49:34.030507 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 67982us
I20260807 19:49:34.039628 8856 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.040704 11314 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.041126 11118 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:34.041961 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.044947 11444 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.076507 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.076928 11545 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.078212 11545 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:34.078687 11545 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.079026 11545 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.045482 11556 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.048449 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:49:34.076485 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.081926 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:34.082487 8902 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:34.048969 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.084384 11533 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.076553 11314 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260807 19:49:34.089627 11314 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:34.051064 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.051863 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.085210 11533 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.094218 11533 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.091581 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.095935 11314 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:34.052802 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.098250 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.053979 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.054294 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.099951 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.054857 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.100741 9020 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:34.052047 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.101336 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.101751 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.055603 11461 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:34.102407 9020 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.060024 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38436 (request call id 195) took 9192 ms. Trace:
I20260807 19:49:34.102850 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.065351 11187 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.067374 11515 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.068298 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:34.068645 11358 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:34.069134 10756 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:34.070226 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 45575.5us
I20260807 19:49:34.070969 11465 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.099917 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.106508 11515 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.104087 11187 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.107020 11515 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.107354 11187 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.105392 11455 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.101279 9008 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:34.065383 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:34.106664 11465 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.074913 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.103184 8630 rpcz_store.cc:276] 0807 19:49:24.867758 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:24.867979 (+ 221us) service_pool.cc:224] Handling call
0807 19:49:34.059694 (+9191715us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:32.153899 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:32.458507 (+304608us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:32.462870 (+ 4363us) write_op.cc:435] Acquired schema lock
0807 19:49:32.462908 (+ 38us) tablet.cc:662] Decoding operations
0807 19:49:32.645246 (+182338us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:32.650857 (+ 5611us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:32.656700 (+ 5843us) tablet.cc:685] Acquiring locks for 9 operations
0807 19:49:32.795538 (+138838us) tablet.cc:701] Row locks acquired
0807 19:49:32.795556 (+ 18us) write_op.cc:260] PREPARE: finished
0807 19:49:32.795725 (+ 169us) write_op.cc:270] Start()
0807 19:49:32.795855 (+ 130us) write_op.cc:276] Timestamp: P: 1786132172795678 usec, L: 0
0807 19:49:32.795873 (+ 18us) op_driver.cc:348] REPLICATION: starting
0807 19:49:32.796606 (+ 733us) log.cc:844] Serialized 3609 byte log entry
0807 19:49:33.562683 (+766077us) op_driver.cc:464] REPLICATION: finished
0807 19:49:33.772696 (+210013us) write_op.cc:301] APPLY: starting
0807 19:49:33.772898 (+ 202us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:33.806034 (+ 33136us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:33.816074 (+ 10040us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:33.828751 (+ 12677us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:33.829157 (+ 406us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0807 19:49:33.829242 (+ 85us) write_op.cc:312] APPLY: finished
0807 19:49:33.829947 (+ 705us) log.cc:844] Serialized 70 byte log entry
0807 19:49:33.830323 (+ 376us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:33.830566 (+ 243us) write_op.cc:454] Released schema lock
0807 19:49:33.830645 (+ 79us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:33.834211 (+ 3566us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:33.834275 (+ 64us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":209480,"apply.run_cpu_time_us":5976,"apply.run_wall_time_us":61022,"num_ops":9,"prepare.queue_time_us":334942,"prepare.run_cpu_time_us":41111,"prepare.run_wall_time_us":1087406,"raft.queue_time_us":87357,"raft.run_cpu_time_us":8837,"raft.run_wall_time_us":176207,"replication_time_us":766571,"thread_start_us":821602,"threads_started":4,"wal-append.queue_time_us":603934}]]}
I20260807 19:49:34.110512 11455 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.112856 11465 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.114619 11465 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:34.103852 11556 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:34.049422 9889 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:34.085170 8901 raft_consensus.cc:2393] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:34.094282 9021 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:34.118666 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.119987 9044 tablet_copy_service.cc:342] P a38f4d6088cf4c8cb510200ca0555801: Request end of tablet copy session 39975243456d4452b4a4870ac7ebafc5-a898110875ba40ab8e512e124aff4fb2 received from {username='slave'} at 127.0.0.1:37380
I20260807 19:49:34.120687 9044 tablet_copy_service.cc:434] P a38f4d6088cf4c8cb510200ca0555801: ending tablet copy session 39975243456d4452b4a4870ac7ebafc5-a898110875ba40ab8e512e124aff4fb2 on tablet a898110875ba40ab8e512e124aff4fb2 with peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:34.119244 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.123296 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.124655 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.123842 9893 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEARNER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:34.123194 11187 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.125300 9021 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:34.126271 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.127120 11187 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.096905 9022 raft_consensus.cc:1240] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:34.113585 10756 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.117309 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:34.117518 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 52278.6us
I20260807 19:49:34.118157 11531 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:49:34.080477 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 50995.5us
I20260807 19:49:34.023274 11055 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.166760 9020 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:34.166913 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.127492 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.138477 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.173777 8901 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:34.114210 11455 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:34.132625 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.171732 11531 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.175590 11055 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.186779 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.172269 11118 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:34.190317 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:34.191082 8915 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:34.170833 11545 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.192677 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.187012 9008 raft_consensus.cc:2393] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:34.192586 8910 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:34.198940 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.204375 9017 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:34.190454 11531 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.200001 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:34.218729 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:34.222268 8911 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:34.219192 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 116533us
I20260807 19:49:34.190811 11055 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.227442 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:34.227661 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.228188 8915 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:34.228744 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 44413.8us
I20260807 19:49:34.230595 11515 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.231106 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.231287 11505 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:34.208218 11455 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.235692 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.236397 9020 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:34.236502 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.192651 11118 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.241608 8740 leader_election.cc:400] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:34.246354 9020 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 19:49:34.232649 11107 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.214085 8904 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:34.247977 8740 leader_election.cc:403] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:34.249657 9022 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:34.252328 11446 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:34.249449 11600 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:34.253799 11600 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.256062 11600 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.261731 9023 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:34.263926 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.270990 9897 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:34.278787 9898 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:34.279906 11524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:37368 (request call id 6) took 10293 ms. Trace:
I20260807 19:49:34.273211 11515 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.225994 11314 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.251415 11455 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.287135 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.290165 9897 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33436 (request call id 131) took 1127 ms. Trace:
I20260807 19:49:34.290638 9897 rpcz_store.cc:276] 0807 19:49:33.163146 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:33.163842 (+ 696us) service_pool.cc:224] Handling call
0807 19:49:34.289912 (+1126070us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54004c4ad4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:34.290093 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434940672}
I20260807 19:49:34.291219 11358 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:34.271826 11533 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.295521 11389 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.304751 10756 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.281570 11606 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260807 19:49:34.290189 11118 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.307883 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.309442 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.300766 11425 consensus_queue.cc:1059] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:34.310268 11606 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:34.311103 11461 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.298027 8968 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:34.313946 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 81662.4us
I20260807 19:49:34.311149 11310 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:49:34.276839 11600 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:34.293438 11331 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.319218 9027 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:34.291664 11358 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:34.322335 11605 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:34.312490 11539 raft_consensus.cc:3055] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
W20260807 19:49:34.325520 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.326920 8911 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:34.308130 11545 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.285524 11309 consensus_queue.cc:1059] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260807 19:49:34.335683 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:34.329998 8737 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:34.320859 11461 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.345644 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.346369 8915 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:34.329941 11331 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.358817 11331 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.326476 11531 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.360337 11055 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.362617 11531 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.344827 10756 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:34.363929 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 54619.9us
I20260807 19:49:34.280391 11524 rpcz_store.cc:276] 0807 19:49:23.980156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:23.980449 (+ 293us) service_pool.cc:224] Handling call
0807 19:49:24.015439 (+ 34990us) log.cc:844] Serialized 499 byte log entry
0807 19:49:34.269361 (+10253922us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81408,"raft.queue_time_us":56843,"raft.run_cpu_time_us":3097,"raft.run_wall_time_us":141437,"thread_start_us":108912,"threads_started":1,"wal-append.queue_time_us":117204,"wal-append.run_cpu_time_us":2316,"wal-append.run_wall_time_us":1003217}
I20260807 19:49:34.333659 11539 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:34.338508 11455 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.313290 11389 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.349699 11461 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.349459 11118 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:34.386430 8779 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 19:49:34.386739 8780 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:34.362571 11055 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.388685 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.393563 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.372378 11446 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:49:34.398020 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 125416us
W20260807 19:49:34.398667 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 53129.7us
I20260807 19:49:34.372002 10756 raft_consensus.cc:697] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:34.394256 8788 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:34.386533 11118 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:34.386288 11107 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.408730 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.397961 11455 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:34.409965 8856 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.411034 8779 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:34.401245 9897 raft_consensus.cc:1240] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:34.408032 11118 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.415674 11600 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260807 19:49:34.419071 11503 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.418706 10756 consensus_queue.cc:237] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.417207 11605 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.419688 10977 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:34.421003 8856 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.421128 8778 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:34.407037 11533 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:34.420104 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.459396 8856 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.453950 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.420725 11606 consensus_queue.cc:1059] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260807 19:49:34.464742 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:34.467079 8779 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:34.454730 11605 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.475957 8787 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:34.479995 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:34.482885 8778 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:34.483579 10988 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:34.484130 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:34.485540 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:34.486934 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 66915.7us
I20260807 19:49:34.468070 11107 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.488652 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:34.495886 8790 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.456168 11503 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:34.500425 11604 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.095s user 0.000s sys 0.003s
I20260807 19:49:34.417527 11539 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.478876 11605 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.506731 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.509938 8968 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.476979 11331 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260807 19:49:34.495299 11107 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:34.513391 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 184673us
I20260807 19:49:34.512594 11486 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.517613 11612 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.518111 11612 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.518517 11612 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.517984 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.520887 11486 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.521437 11486 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.520316 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.508801 11314 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.521824 11486 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.529207 11324 raft_consensus.cc:3055] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:34.530081 11465 raft_consensus.cc:3055] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:34.533336 11055 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:34.513806 11539 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.533888 11444 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260807 19:49:34.537014 11503 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.522634 11107 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:34.538450 11610 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260807 19:49:34.520721 11600 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.511874 11082 consensus_queue.cc:1059] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260807 19:49:34.551070 11055 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.573021 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.547042 11324 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:34.549230 11465 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:34.553586 11533 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260807 19:49:34.587193 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 77354.4us
I20260807 19:49:34.592386 11465 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.594063 11465 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.597059 11324 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.574956 11055 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:34.610055 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.624434 11358 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.625847 11358 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:34.631176 11556 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:34.631678 11556 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.631995 11556 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:34.643070 11389 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.647287 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:34.647606 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.648048 8911 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
W20260807 19:49:34.648530 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.649232 11331 raft_consensus.cc:3055] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:34.649648 11331 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:34.650405 9027 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:34.653191 11466 raft_consensus.cc:3055] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:49:34.653661 11466 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:34.654688 11466 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:34.657372 11598 log.cc:927] Time spent T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.395s user 0.002s sys 0.000s
I20260807 19:49:34.661518 11608 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:34.661983 11608 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:34.662859 11608 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.663005 11612 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.664686 9850 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:34.650501 11331 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:34.683962 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 198910us
I20260807 19:49:34.684602 8790 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:34.656143 11466 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
W20260807 19:49:34.691627 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 232325us
I20260807 19:49:34.692785 8856 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.695395 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.694535 11503 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:34.685375 11505 consensus_queue.cc:1175] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a38f4d6088cf4c8cb510200ca0555801" responder_term: 5 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 19:49:34.720551 10977 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.720841 11331 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:34.724938 8714 auto_rebalancer.cc:644] Failed to schedule move for tablet ed0dd3f602c145219dc8717e182ff568: Illegal state: Replica a38f4d6088cf4c8cb510200ca0555801 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:34.727296 11515 raft_consensus.cc:3055] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:34.727777 11515 raft_consensus.cc:740] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:34.697168 11455 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:34.728691 11455 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.729031 11455 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.730160 11324 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.724562 11055 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:34.743582 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 59225.4us
I20260807 19:49:34.748515 10988 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.749161 10988 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:34.752501 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 57173.2us
I20260807 19:49:34.753907 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:34.755399 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.695966 11389 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:34.752281 11601 consensus_queue.cc:1059] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260807 19:49:34.787083 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:34.789347 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 44615.5us
I20260807 19:49:34.751392 11600 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:34.760914 11389 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:34.761370 11608 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.751451 10988 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.762068 10977 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:34.793169 11055 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:34.798089 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:34.803159 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 352052us
W20260807 19:49:34.807147 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 192437us
I20260807 19:49:34.763945 11630 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260807 19:49:34.809422 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 50126.9us
I20260807 19:49:34.796783 11606 raft_consensus.cc:3055] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:34.797724 11389 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:34.802062 11533 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:49:34.816042 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.824957 11629 raft_consensus.cc:1064] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: attempting to promote NON_VOTER 39975243456d4452b4a4870ac7ebafc5 to VOTER
I20260807 19:49:34.825709 9021 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 19:49:34.797756 11107 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.828310 8780 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:34.829452 11107 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.832012 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:34.833133 8789 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:34.831291 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:34.838418 8778 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:49:34.827556 11629 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.810613 11630 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.813268 11606 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:34.819398 10756 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 19:49:34.848124 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:34.849048 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.799841 11515 consensus_queue.cc:260] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:34.871368 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 58535.5us
I20260807 19:49:34.859207 11533 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:34.877365 11533 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.880981 11622 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:34.881593 11622 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:34.882473 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.847864 11612 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.844306 11630 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:34.885727 11515 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:34.873819 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:34.889266 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.889907 11486 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.891399 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.891597 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:34.892076 9022 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:34.892474 11612 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.892848 8779 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:34.893019 11612 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.893566 11539 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:34.894038 11539 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.894411 11539 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:34.895911 11612 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.891399 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:34.897030 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.897962 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:34.899849 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.884092 11622 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.848475 11415 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:34.911401 11415 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.912035 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:34.912858 9019 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:34.893174 11524 raft_consensus.cc:3055] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:34.859241 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:34.859802 11314 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.856179 11503 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.844928 11461 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.914151 11524 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:34.870949 10756 raft_consensus.cc:697] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:34.905400 10977 raft_consensus.cc:697] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:34.944674 11641 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:34.945194 11641 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.945583 11641 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:34.953089 11644 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:49:34.953953 11631 log.cc:927] Time spent T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.144s user 0.001s sys 0.000s
I20260807 19:49:34.934814 11556 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.921253 11314 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:34.924608 11503 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:34.960781 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.933312 11486 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:34.925611 11622 consensus_queue.cc:1048] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
W20260807 19:49:34.962201 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 74725.2us
I20260807 19:49:34.981364 8856 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:34.982304 11486 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:34.963740 11606 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.984843 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:34.977800 11118 consensus_meta.cc:287] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.524s user 0.010s sys 0.001s
I20260807 19:49:34.985819 11612 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:34.986254 11612 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:34.977677 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 105451us
I20260807 19:49:34.964074 11314 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:34.992862 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:34.985841 11118 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:34.995707 11118 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:34.992590 11647 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:34.986593 11612 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:34.998692 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:35.002063 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:34.931138 11605 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:34.954771 9008 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:34.961102 11644 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.018967 11655 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:35.019658 11655 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.020179 11655 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:35.026820 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.015643 11644 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.048116 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.031935 11649 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260807 19:49:35.050959 11649 raft_consensus.cc:3055] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:35.051313 11649 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:35.051402 11389 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:35.052253 11649 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.053975 11649 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.057296 11630 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.046284 10977 consensus_queue.cc:237] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.059695 11630 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.060637 11657 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:35.061069 11657 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:35.061260 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.061456 11657 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
W20260807 19:49:35.064328 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.068764 11314 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.073287 11649 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:35.073810 11649 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.080631 11649 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.081080 11461 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.073175 9893 raft_consensus.cc:2368] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:35.083464 11455 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.084980 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.085659 9027 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:35.073889 8849 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.080828 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.087599 8911 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:35.088475 8856 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:35.099495 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.100961 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.102205 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.068764 11415 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:35.104247 11623 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:35.104795 8787 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.104723 11623 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.105923 11649 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.106628 11623 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.107208 8787 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.108062 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.103211 11461 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:35.110021 11461 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:35.110392 11461 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.115862 11623 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.116745 9889 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEARNER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 19:49:35.117658 11612 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.073977 11389 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:35.107285 11556 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:35.121815 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.122076 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.121202 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.122449 9023 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:35.127137 8856 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.129212 11358 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:35.129731 11358 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.130098 11358 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.118604 11415 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260807 19:49:35.108184 11649 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:35.149739 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 48894.6us
I20260807 19:49:35.152808 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.153825 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.157110 11612 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.157549 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.158361 8906 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:35.160182 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.161841 8915 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.152829 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.164153 8904 raft_consensus.cc:2393] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:35.165149 11314 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.156985 8902 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:35.145093 8778 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:35.144160 11539 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.032672 10756 consensus_queue.cc:237] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.181471 8790 raft_consensus.cc:1275] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:35.181926 8776 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:35.187438 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 60398.4us
W20260807 19:49:35.188136 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 263960us
I20260807 19:49:35.175196 8910 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:35.188956 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:35.192862 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.220145 9018 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.220434 11455 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.220801 8776 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 19:49:35.223254 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.195990 11655 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.231021 11503 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.179823 11641 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.196412 11622 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:35.122782 11623 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.197850 11331 consensus_queue.cc:1059] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:35.199098 11444 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:35.233521 11655 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.233521 11331 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:35.234092 11331 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.234946 11331 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.194100 11664 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:35.201915 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:35.239800 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 50915.9us
W20260807 19:49:35.203917 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.203840 8906 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:35.212118 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:35.243270 8972 leader_election.cc:336] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.544s (SENT)
I20260807 19:49:35.212492 8911 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:35.191265 10988 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.179867 11539 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.193596 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.245191 10988 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.224263 8737 leader_election.cc:400] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Vote denied by peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143) with higher term. Message: Invalid argument: T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:35.246138 8737 leader_election.cc:403] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260807 19:49:35.245780 11657 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.231099 11644 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.247963 8903 raft_consensus.cc:2410] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:35.248468 11660 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:35.248960 11660 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.249332 11660 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.225948 11627 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 19:49:35.227741 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.230120 11466 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:35.231313 11055 consensus_queue.cc:1059] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:35.201434 11465 consensus_queue.cc:1059] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:35.250064 8790 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.236146 11331 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.248075 11657 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.253605 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:35.259982 9850 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:35.267729 11539 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:35.268246 11539 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.268651 11539 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.272362 11605 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.274365 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.247864 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.265152 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:35.252950 11486 raft_consensus.cc:3060] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:35.251888 11629 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.217756 11461 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.246260 11664 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.265249 11389 raft_consensus.cc:2804] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:35.304360 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:35.307147 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.322381 9020 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.308032 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.255409 11455 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.309413 11107 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:35.312119 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.302266 8904 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:35.319268 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.334844 9019 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.258157 11622 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:35.343703 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:35.347946 8915 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:35.271080 8630 catalog_manager.cc:5697] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.349519 8901 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:35.357641 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:35.365477 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.373029 8903 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.373531 11664 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.334084 11107 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.345237 11630 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:35.346431 11389 raft_consensus.cc:697] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:35.316962 11629 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.377997 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:35.335814 11455 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.354028 8630 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 3 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:35.383086 9022 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:35.354908 11503 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.386066 8856 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.363691 11533 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:35.377839 11107 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.381969 11664 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:35.386026 8630 catalog_manager.cc:5697] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 3 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:35.405212 9020 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:35.405802 8902 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:35.401346 8630 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 4 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:35.407438 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.407596 8630 catalog_manager.cc:5697] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 4, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.411341 9018 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:35.391899 11623 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:35.390194 10977 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:35.388700 11533 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.413056 8856 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.381469 11358 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.416177 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.440095 8783 raft_consensus.cc:2393] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:35.418220 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:35.419378 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:35.420163 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.390339 11622 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.421228 8630 catalog_manager.cc:5697] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3, now has a pending config: VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193), VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194), VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195), NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:35.423008 10988 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:35.424319 11606 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.448307 11358 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.443789 10977 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:35.449527 11455 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.408135 11641 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.449699 10977 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.454267 11465 raft_consensus.cc:3055] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:35.454732 11629 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:35.413164 11524 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.441980 11608 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:35.425474 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:35.456938 11608 raft_consensus.cc:697] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:35.457825 11608 consensus_queue.cc:237] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:35.427802 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:35.431995 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 195075us
I20260807 19:49:35.462919 8910 raft_consensus.cc:2410] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260807 19:49:35.463564 8968 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.467082 10988 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:35.467646 10988 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.439455 11533 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.468616 10988 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.471163 11415 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:35.442103 11623 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:35.473006 11415 consensus_queue.cc:1059] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:35.407984 11389 consensus_queue.cc:237] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:35.474722 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.475539 8788 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:35.478044 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.434621 11627 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:35.413110 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.438902 9019 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:35.411252 11605 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:35.444031 8630 catalog_manager.cc:5697] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.484781 11605 raft_consensus.cc:697] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:35.485147 8630 catalog_manager.cc:5697] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 4, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.486580 8630 catalog_manager.cc:5697] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.487907 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.488636 8788 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:35.488387 8630 catalog_manager.cc:5697] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 5, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.487943 11605 consensus_queue.cc:237] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.490183 8787 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.489780 8630 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 3, leader changed from 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 3 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:35.488945 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.491438 8775 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:35.491863 8790 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:35.491473 8630 catalog_manager.cc:5697] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.494544 8630 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: config changed from index 2 to 3, term changed from 1 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to <none>, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:35.498128 8630 catalog_manager.cc:5697] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:35.500905 8630 catalog_manager.cc:5697] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.502100 11486 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143) with higher term. Message: Invalid argument: T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:35.505092 11623 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.511857 8630 catalog_manager.cc:5697] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 3 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:35.454726 11465 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.506775 8783 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.513741 8630 catalog_manager.cc:5697] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.413810 11664 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:35.517024 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:35.517711 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:35.518430 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.520720 8903 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:35.486532 11627 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.522433 8779 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:35.523485 11446 raft_consensus.cc:3055] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:35.523909 11446 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.518344 8630 catalog_manager.cc:5697] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 2, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 2 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:35.524866 11446 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.525197 8630 catalog_manager.cc:5697] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.526516 8630 catalog_manager.cc:5697] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 4, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.522394 11627 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.529551 11627 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.527936 8630 catalog_manager.cc:5697] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:35.477203 11524 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.539114 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:35.534814 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.549365 11465 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.550753 11465 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.552294 8856 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:35.552987 11668 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260807 19:49:35.554011 11664 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.531878 8630 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:35.556991 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.557446 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:35.557126 8630 catalog_manager.cc:5697] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:35.558383 11657 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:35.558983 11657 raft_consensus.cc:697] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:35.561868 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.563105 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.564079 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.565930 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.567399 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.557772 8788 raft_consensus.cc:2368] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:35.559898 11657 consensus_queue.cc:237] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.569799 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:35.571183 11107 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.572587 11446 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.573006 11466 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:35.573657 11466 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.562970 11461 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.577121 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.566089 11644 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.578336 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.587999 8903 raft_consensus.cc:1240] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:35.577823 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 57742.2us
W20260807 19:49:35.589582 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:35.592305 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.598031 8968 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.599107 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.603415 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.604097 8910 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:35.610075 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.610951 8901 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:35.614331 8915 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.617187 9021 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:35.618566 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.621254 8968 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.625070 9021 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.628574 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.629666 8911 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:35.631978 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.635353 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.643723 9027 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:35.646638 8790 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.500126 11533 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:35.652824 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.620072 11545 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:35.653349 9838 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:35.656491 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.661734 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.642364 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.665947 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.666291 8780 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.667467 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.675558 8915 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.655061 11629 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:35.669937 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.651733 11055 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:35.668892 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.678848 9838 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.681197 9018 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 19:49:35.681715 11486 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:35.681996 11644 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:35.682490 11457 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260807 19:49:35.695336 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.605265 11660 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.658396 11545 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:35.724453 8913 raft_consensus.cc:2393] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:35.655352 11628 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260807 19:49:35.728521 8915 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:35.730221 8908 raft_consensus.cc:2368] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:35.651283 10756 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260807 19:49:35.738346 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.744892 8902 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:35.733938 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:35.721256 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.683957 11065 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260807 19:49:35.748942 11107 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.750178 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.752264 8788 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:35.733175 11389 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.723246 11629 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:35.736789 11545 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:35.757870 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.762934 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.765470 8908 raft_consensus.cc:2393] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:35.723492 11055 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:35.743296 11505 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:35.768867 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.765053 8913 raft_consensus.cc:2376] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:35.762871 11628 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:35.736964 11486 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:35.815119 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 46330.8us
W20260807 19:49:35.793455 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 138162us
I20260807 19:49:35.765928 11107 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:35.765040 11629 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.826268 9027 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:35.816027 11664 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:35.815161 11628 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.820724 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.735929 11644 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:35.827664 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 148912us
I20260807 19:49:35.821165 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.876775 9838 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:35.877642 8906 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260807 19:49:35.817657 11657 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.818531 11107 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.884963 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:35.892009 9027 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:35.886559 11682 log.cc:927] Time spent T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
I20260807 19:49:35.889786 8775 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:35.891299 9017 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:35.830513 10988 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.829319 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.899580 11466 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:35.904815 11681 log.cc:927] Time spent T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.155s user 0.000s sys 0.003s
I20260807 19:49:35.927541 8787 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.905217 8913 raft_consensus.cc:1240] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: [1.2-3.3]
W20260807 19:49:35.905471 11679 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.156s user 0.000s sys 0.001s
W20260807 19:49:35.910707 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 90044.9us
I20260807 19:49:35.891458 11657 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.912608 8790 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.913228 11630 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.913534 11118 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.913805 11486 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.883967 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.884323 11627 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.915026 8903 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.918181 11082 raft_consensus.cc:3055] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
W20260807 19:49:35.918715 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.919301 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:35.922266 11466 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:35.930164 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.931704 9018 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:35.930987 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:49:35.932091 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 55409.5us
I20260807 19:49:35.876981 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.936713 11630 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:35.937042 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.896651 11107 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:35.942602 8906 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.913182 11641 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:35.953234 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.944274 8903 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.940905 11082 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:35.945436 8904 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:35.945971 11660 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.947602 8913 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:35.949374 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.949384 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:49:35.950531 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:35.958317 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:35.950943 9015 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:35.934122 11503 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.938580 11688 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:35.956175 11486 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:35.952649 11457 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:35.965052 11457 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:35.967511 11457 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.968653 11055 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:35.938719 11331 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 19:49:35.971518 8903 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.933036 10988 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:35.971797 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:35.974311 8906 raft_consensus.cc:2410] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260807 19:49:35.974866 8913 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:35.973270 11660 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:35.977999 11660 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:35.978425 11660 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.980046 8776 raft_consensus.cc:1240] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.981545 11644 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.984200 11641 consensus_queue.cc:1059] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:35.985260 11641 raft_consensus.cc:3055] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:35.985610 11641 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:35.986487 11641 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.987524 11641 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.943727 11118 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:35.989653 11539 consensus_meta.cc:287] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.721s user 0.016s sys 0.006s
I20260807 19:49:35.990111 11118 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:35.944073 11627 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.953868 11628 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:35.991616 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.994032 8915 raft_consensus.cc:1240] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:35.991828 11628 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:35.996837 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:35.995858 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:35.997674 8903 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:35.969448 11630 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:35.972209 11486 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:36.012979 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:36.012458 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.015970 9893 raft_consensus.cc:2393] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:36.014420 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:35.971967 11503 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.002369 10977 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:36.002611 11620 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.002911 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:36.000039 11118 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:36.004974 11533 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.013372 11486 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.996865 11644 consensus_queue.cc:1059] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260807 19:49:36.077456 8903 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.014267 11082 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.017940 11630 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:36.079970 9889 raft_consensus.cc:2393] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:36.081533 11503 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:35.997476 11539 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.082074 8904 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:36.085186 11641 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
W20260807 19:49:36.086354 11465 consensus_meta.cc:287] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.535s user 0.010s sys 0.002s
I20260807 19:49:36.099680 11505 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.100960 11522 raft_consensus.cc:3055] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:36.079490 11620 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:36.102494 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.079174 11118 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:36.104684 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:36.155849 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:36.156975 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:36.157362 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.015255 11627 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:36.158301 8910 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:36.018818 9897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
W20260807 19:49:36.148530 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 134142us
I20260807 19:49:36.147369 11082 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:35.978847 11444 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:36.160647 8778 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:36.146400 11657 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.146870 11545 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.155495 11630 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:36.167793 11630 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.151141 11389 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:36.157977 11118 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.160583 11505 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.175928 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.179039 8901 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:36.180934 9023 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:36.159471 11664 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:36.155632 11522 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.161441 11622 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.166594 11539 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.190054 8901 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260807 19:49:36.198952 11683 log.cc:927] Time spent T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.449s user 0.004s sys 0.013s
I20260807 19:49:36.202800 11505 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.218406 11629 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.220841 11522 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.222441 11522 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:36.182967 11664 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:36.228948 10988 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:36.234233 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.235533 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.246352 8901 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:36.236279 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.188738 11107 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.243669 11465 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:36.250130 11465 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:36.250248 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.250545 11465 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.251718 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.252399 8776 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:36.244465 11627 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:36.253278 11627 raft_consensus.cc:3055] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:36.253682 11627 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.247810 8740 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.254583 11627 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.262867 11503 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:36.251421 11465 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.264001 11389 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:36.266676 11539 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.267387 11539 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.271265 11664 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.272835 11664 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:36.276175 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:36.278424 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.280728 9008 raft_consensus.cc:1275] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.281198 11331 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.281749 11331 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.283773 9023 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260807 19:49:36.284868 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.295034 8910 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:36.299893 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.300673 8910 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:36.305236 8903 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.306053 11461 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.306587 11461 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:36.307349 11674 log.cc:927] Time spent T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.572s user 0.003s sys 0.006s
I20260807 19:49:36.191603 11622 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.313810 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:36.314160 11608 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:36.314508 9008 raft_consensus.cc:606] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:36.314916 9008 raft_consensus.cc:614] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:36.235816 11641 raft_consensus.cc:697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:36.322836 8915 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:36.317405 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.327263 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.316232 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.332208 8908 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:36.332418 8783 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:36.335407 11545 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:36.322983 8780 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260807 19:49:36.338260 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.321980 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.324066 8788 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:36.339538 8910 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:36.340005 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.341449 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:36.342402 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.342366 9889 raft_consensus.cc:2393] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:36.341627 9897 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:36.343602 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:36.344244 9893 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.346458 11107 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.348140 8910 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:36.320202 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.349364 11118 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.350473 8970 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.350665 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.351253 11539 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.351305 11545 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:36.349956 11620 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.351918 11545 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:36.352176 11629 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:36.352408 11545 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:36.352563 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:36.353351 9893 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:36.354728 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.353389 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.355374 9018 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:36.355564 9015 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:36.356915 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:36.357899 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.359941 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.360632 9008 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.361353 11545 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.362685 9023 raft_consensus.cc:2393] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:36.357018 8970 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.364205 11620 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.364848 11620 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.365125 9018 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:36.366690 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.367491 11620 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:36.367684 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.368310 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.368666 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.369124 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.368217 11620 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:36.371171 11620 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:36.371582 11620 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.383249 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.365175 11627 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:36.385329 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.365123 11118 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.385679 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.386916 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.392251 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.393656 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.367437 11486 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.430325 8902 raft_consensus.cc:2376] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:36.397979 11660 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:36.435418 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 9d0dfd76f4dc484499bc12ddf5871c48 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:36.379794 11641 consensus_queue.cc:237] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.402328 8908 raft_consensus.cc:2410] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260807 19:49:36.402814 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.442775 9022 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:36.404187 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.443295 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.443854 9020 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:36.420400 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.424384 8778 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:36.422041 8856 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.446985 11623 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:36.447362 11623 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:36.447696 11623 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:36.449687 8915 raft_consensus.cc:1240] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:36.450402 11486 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.451012 11545 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.450786 9023 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:36.451785 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.452706 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.453531 8903 raft_consensus.cc:2410] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:36.458618 8901 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:36.437054 11465 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.437979 8902 raft_consensus.cc:1275] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260807 19:49:36.431274 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.440095 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.441298 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.463415 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.463810 8790 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.464306 8902 raft_consensus.cc:2376] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3. Re-sending same reply.
I20260807 19:49:36.464421 11657 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:36.465318 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:36.466149 11660 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.465989 11657 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:36.441815 9015 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:36.465951 9893 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:36.463778 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:36.470371 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.450693 11457 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:36.485019 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.486038 8903 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:36.487006 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.487280 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.487838 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.488544 8915 raft_consensus.cc:2376] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4. Re-sending same reply.
I20260807 19:49:36.489629 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.490033 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.490630 8906 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.488657 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.495785 8790 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.496327 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.445381 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.492785 8908 raft_consensus.cc:2376] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4. Re-sending same reply.
I20260807 19:49:36.461814 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.428952 11608 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.493202 8913 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:36.444938 11055 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260807 19:49:36.522264 9893 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:36.444777 11466 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
W20260807 19:49:36.523569 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 62245.3us
I20260807 19:49:36.477610 11461 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.528426 8911 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260807 19:49:36.529071 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 57076.1us
I20260807 19:49:36.532358 8911 pending_rounds.cc:85] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 2
I20260807 19:49:36.534705 9015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afb6953f02d1486aaee40c8901954290"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:36.534850 8970 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.513451 11486 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.478432 8775 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.535440 9015 raft_consensus.cc:606] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:36.481465 11457 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:36.544767 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:36.538106 8911 pending_rounds.cc:107] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 19:49:36.545637 8911 raft_consensus.cc:2889] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:36.546802 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:36.548084 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.549336 11331 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.551940 11622 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.558775 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.560767 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.560425 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.540792 11486 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:36.563429 8972 leader_election.cc:336] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.514s (SENT)
W20260807 19:49:36.567986 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:36.564107 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.560081 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.569684 8901 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.564591 8913 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:36.564328 8910 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:36.564788 8903 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.560856 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.571907 8908 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:36.489595 11623 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.526844 8904 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:36.591030 8901 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.542322 11055 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:36.565295 11457 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:36.604054 11055 raft_consensus.cc:697] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.610284 11055 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.607532 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:36.612941 9889 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:36.614347 8970 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:36.620515 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.620826 11486 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.621352 11608 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:36.621675 11628 consensus_meta.cc:287] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.619s user 0.014s sys 0.000s
I20260807 19:49:36.624779 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:36.625676 9008 raft_consensus.cc:606] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:36.626082 9008 raft_consensus.cc:614] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:36.626930 11331 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:36.627388 11331 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:36.627748 11331 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:36.628909 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 381164us
I20260807 19:49:36.635968 11331 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.636503 8776 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:49:36.638073 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.638300 11629 raft_consensus.cc:3055] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:36.638768 11629 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.640252 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:36.640944 9889 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:36.641297 11699 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.310s user 0.001s sys 0.000s
I20260807 19:49:36.643564 11620 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.644969 9017 raft_consensus.cc:2368] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:36.624886 11623 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.647101 11620 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.647293 11486 consensus_queue.cc:1059] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:36.647574 9018 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:36.648269 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:36.648499 11486 raft_consensus.cc:3055] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:36.648974 9898 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:36.657029 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:36.657684 8972 leader_election.cc:336] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.526s (SENT)
I20260807 19:49:36.659057 9898 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.647109 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.662721 9027 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:36.664638 8908 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:36.639708 11629 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.659934 9019 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:36.660676 8970 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.668519 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.670899 11664 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:36.670984 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 140809us
I20260807 19:49:36.671736 8856 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.672583 11623 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:36.672983 11623 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:36.676797 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet d8160be756c34002a2ba00cf8deaa10a (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:36.673173 8740 leader_election.cc:400] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:36.674746 9020 raft_consensus.cc:1275] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:36.674746 9023 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.676085 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.673336 11623 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.676780 11660 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:36.688685 11082 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:36.689311 11082 raft_consensus.cc:697] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.689604 11660 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:36.690052 11660 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:36.690573 11629 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.649003 11486 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:36.692237 11486 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.693531 11486 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:36.697036 11082 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.697679 11457 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.699751 9893 raft_consensus.cc:2393] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 7: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:36.699926 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.700613 8778 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:36.703913 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.704541 8776 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.705331 11623 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.706079 9889 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.706449 8902 raft_consensus.cc:2368] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:36.707412 8970 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.708716 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:36.709014 8972 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.607s (SENT) [suppressed 22 similar messages]
I20260807 19:49:36.709352 8788 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.710929 9023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:36.668913 11331 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:36.681165 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.713883 11118 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:36.717187 8788 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.717473 11664 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:36.719730 11539 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:36.721807 11118 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:36.722007 11465 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:36.723291 11620 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:36.723290 8968 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:36.728134 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.728794 9838 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.731998 8776 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:36.735687 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.739765 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:36.746171 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.746690 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.747112 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.741204 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.747717 8779 raft_consensus.cc:2468] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:36.742405 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.748670 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.748914 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.749513 8787 raft_consensus.cc:2410] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:36.749676 8788 raft_consensus.cc:2393] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.746104 9020 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:36.750087 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.749347 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.747869 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.751391 8778 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.751535 8902 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.752869 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.754172 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.753302 11688 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:36.755381 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.756098 9020 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.757776 11660 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:36.744130 8849 connection.cc:256] RPC call timeout handler was delayed by -1.135453490074724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17220000000000002s for a response
I20260807 19:49:36.761085 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.761901 8783 raft_consensus.cc:2393] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.759953 11660 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.764492 8968 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.766779 9020 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:36.767611 11331 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 19:49:36.770406 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.762301 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.773444 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.774083 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.774854 8910 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:36.776784 9020 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.776952 8903 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.768141 11465 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.778756 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.744120 11107 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.780418 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.803732 9027 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:36.778981 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.804944 8789 raft_consensus.cc:2410] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:36.781296 9023 raft_consensus.cc:606] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.778429 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.805821 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.806187 8910 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.808074 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:36.806205 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.808712 9020 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:49:36.809670 8968 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.792074 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.797549 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.769039 11503 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.762647 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.799842 11457 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.745397 11695 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.800499 8913 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260807 19:49:36.800756 11605 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:36.801944 11465 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.778379 11628 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:36.803160 11623 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.812503 11522 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:36.812909 11082 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.803738 11107 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.814181 8902 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:36.814440 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.815434 11461 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:36.818473 9022 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
W20260807 19:49:36.825322 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.825580 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:36.846398 8911 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:36.826319 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.815217 8856 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.829406 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.849650 8776 raft_consensus.cc:2393] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.830017 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:36.829944 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.832578 9020 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:49:36.822921 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.850874 8780 raft_consensus.cc:2393] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.851377 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.834635 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:36.852398 9019 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:36.833531 11515 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:36.838351 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:36.836608 8910 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:36.809803 11331 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.842376 8904 raft_consensus.cc:2393] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:36.815181 11608 consensus_queue.cc:1048] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
I20260807 19:49:36.837539 8972 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.854112 9022 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:36.856983 8789 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.856931 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:36.857904 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.859233 8783 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:36.860026 9020 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:36.860016 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.841625 11522 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:36.842195 11055 consensus_queue.cc:1059] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260807 19:49:36.845001 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.840548 11695 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.828876 11503 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.864651 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.848091 11082 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.845264 11628 raft_consensus.cc:697] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.858114 11457 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:36.890784 8780 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.885597 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.875761 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.891722 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.884683 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.908321 8787 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:36.908938 8776 raft_consensus.cc:2393] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 7: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:36.893499 9021 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:36.875061 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.873528 8972 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.910147 8790 raft_consensus.cc:2393] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.910173 8788 raft_consensus.cc:2393] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.911283 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:36.910245 8778 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
W20260807 19:49:36.895620 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 65768.7us
I20260807 19:49:36.909572 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.912423 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.912541 8849 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.913156 8787 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:36.914903 11503 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.915311 8779 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:36.918001 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.924796 8789 raft_consensus.cc:2410] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:36.924914 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.925834 8790 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:36.921931 8911 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:49:36.926121 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.926972 11713 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.927462 11713 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.867122 11524 consensus_queue.cc:1059] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260807 19:49:36.867583 11623 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:36.928928 11623 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.930629 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:36.931344 8783 raft_consensus.cc:2393] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:36.873852 11627 consensus_queue.cc:1059] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.059s
W20260807 19:49:36.933506 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60084.9us
I20260807 19:49:36.934078 11713 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.935777 8972 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.936566 10988 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:36.864262 11620 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:36.880295 11065 raft_consensus.cc:3055] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:36.937965 11065 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.938530 11627 raft_consensus.cc:3055] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:36.938930 11627 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.875563 11107 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.930447 8789 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:36.939869 11627 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.942598 11545 consensus_queue.cc:1059] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:36.943495 11695 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.943653 11545 raft_consensus.cc:3055] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:36.944047 11627 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:36.945313 11082 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:36.947052 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.937206 11623 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:36.937276 10988 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260807 19:49:36.944025 11545 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.948755 11623 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:36.869128 11331 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.949368 11545 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.950461 11524 raft_consensus.cc:3055] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:36.951390 8789 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:36.938884 11065 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.948143 11620 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:36.960359 9020 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:36.950932 11524 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.948096 8972 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.867942 11522 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:36.867018 11720 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:36.962779 8790 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:36.972621 8776 raft_consensus.cc:1275] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260807 19:49:36.880457 11657 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:36.972833 8783 raft_consensus.cc:1275] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.971549 8788 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:36.973322 8776 pending_rounds.cc:85] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:36.973307 8790 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:36.973801 8776 pending_rounds.cc:107] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 19:49:36.974090 8776 raft_consensus.cc:2889] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:36.976701 11389 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:36.977304 11389 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:36.977715 11741 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:36.978708 11741 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:36.979611 11741 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:36.977428 8972 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:36.982209 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.982810 8787 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:36.983660 8790 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:36.986275 8968 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:36.989799 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 78577.7us
I20260807 19:49:36.991552 8972 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:36.991477 8856 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:37.005584 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:36.959627 11457 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:36.962590 11713 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:36.972277 11720 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:37.008270 8856 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:37.074929 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 66769us
I20260807 19:49:36.990837 11055 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:36.873771 11629 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.115s
I20260807 19:49:37.004032 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:37.078370 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:37.078471 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:37.084249 9021 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:49:36.991564 11545 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:36.980157 11389 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.087067 8856 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:36.991048 11082 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:37.089063 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 97594.2us
I20260807 19:49:37.092590 8972 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.050237 11623 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:37.050755 11466 raft_consensus.cc:2764] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260807 19:49:37.051074 11461 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:37.076638 11720 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:37.090025 11065 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:37.083503 11657 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 19:49:37.107947 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 152910us
I20260807 19:49:37.094388 11620 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:37.117520 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:37.118441 9021 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:37.129288 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 216815us
I20260807 19:49:37.131438 8849 leader_election.cc:400] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:37.132026 8849 leader_election.cc:403] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:37.136118 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.137166 8911 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:37.075793 11628 consensus_queue.cc:237] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.115753 11629 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:37.110597 11461 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:37.102133 11623 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.090541 11524 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:37.161651 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 175466us
I20260807 19:49:37.150425 11515 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:37.169822 11461 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.170290 8629 catalog_manager.cc:5697] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:37.165794 11515 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:37.173077 11515 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:37.181432 11608 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 4
I20260807 19:49:37.185993 8783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fced846726734919ac0881f9a26379ba"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:60338
W20260807 19:49:37.186327 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 182381us
I20260807 19:49:37.186698 11695 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:37.187183 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.145182 11457 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:37.188717 8783 raft_consensus.cc:606] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.189152 8783 raft_consensus.cc:614] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 19:49:37.190585 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet fced846726734919ac0881f9a26379ba (from 8e9ad2b5e2a94d4faff90af735c6d792 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:37.191125 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260807 19:49:37.192746 11695 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 19:49:37.191046 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:37.193351 11695 raft_consensus.cc:697] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.194147 11605 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:37.197153 11620 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:37.197764 11620 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:37.194353 11695 consensus_queue.cc:237] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.204845 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.211634 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
W20260807 19:49:37.211457 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 89402.3us
I20260807 19:49:37.154309 11657 raft_consensus.cc:697] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:37.216290 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:37.222983 8629 catalog_manager.cc:5697] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added, no longer has a pending config: VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193), VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194), VOTER a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195), NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
I20260807 19:49:37.225931 11524 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:37.225417 8629 catalog_manager.cc:5697] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 2 to 3, leader changed from 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:37.227659 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:49:37.228688 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
W20260807 19:49:37.229175 11746 log.cc:927] Time spent T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.062s user 0.000s sys 0.002s
I20260807 19:49:37.233786 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:37.210681 11664 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 19:49:37.235889 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 148898us
I20260807 19:49:37.226990 8629 catalog_manager.cc:5697] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 3 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 5 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:37.247527 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.248790 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.249915 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:37.213212 11503 raft_consensus.cc:2764] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260807 19:49:37.211730 11107 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 19:49:37.231709 11688 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:37.232474 11623 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.213243 11660 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260807 19:49:37.264096 8740 leader_election.cc:400] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:37.264560 8740 leader_election.cc:403] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:37.256770 11486 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:37.258386 11118 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:37.259472 11539 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 19:49:37.264788 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 133526us
I20260807 19:49:37.270097 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:37.261834 11664 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.265084 11503 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:37.288714 8972 leader_election.cc:336] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659): Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 1.883s (SENT)
I20260807 19:49:37.289307 8972 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.290150 11628 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:37.266335 11486 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:37.293131 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:37.298520 11486 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:37.300416 11750 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.136s user 0.000s sys 0.003s
I20260807 19:49:37.302547 11515 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.305238 11745 raft_consensus.cc:2764] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260807 19:49:37.308147 8972 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.309173 11627 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:37.309662 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.312618 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:37.291841 11620 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.287952 11522 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.305382 11055 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:37.326933 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.328128 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:37.328918 8901 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:37.330920 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.331288 9027 raft_consensus.cc:2368] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:37.331658 8911 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:37.334718 11620 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:37.335212 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.273823 11657 consensus_queue.cc:237] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.336069 8904 raft_consensus.cc:2376] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7. Re-sending same reply.
W20260807 19:49:37.336534 11751 log.cc:927] Time spent T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.152s user 0.003s sys 0.000s
I20260807 19:49:37.340603 11664 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.341485 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:37.342805 11539 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.342693 11630 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:37.346789 9893 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:49:37.308037 11524 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.016s
W20260807 19:49:37.342685 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 79253.3us
I20260807 19:49:37.349241 8901 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:37.350560 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.350672 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:37.328006 8903 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:37.351539 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.318027 11444 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:37.321099 9889 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 19:49:37.355547 8911 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:37.356176 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:37.356978 8778 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:37.357797 8780 raft_consensus.cc:1275] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:49:37.297739 11743 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:37.354429 8972 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.360311 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:37.360368 8972 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.362284 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.372918 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.374145 9897 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260807 19:49:37.375854 11107 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:37.376449 11503 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:37.370882 11641 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260807 19:49:37.379881 11545 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:37.343848 11539 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.394136 8779 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 6
I20260807 19:49:37.325142 11055 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:37.399181 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:37.396479 8629 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:37.407454 11515 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.407855 11545 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:37.413789 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:37.363729 11630 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:37.356452 11444 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:37.428560 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 55721us
I20260807 19:49:37.429857 8972 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:37.430106 9015 raft_consensus.cc:1275] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:37.433094 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38426 (request call id 207) took 3068 ms. Trace:
I20260807 19:49:37.434121 8630 rpcz_store.cc:276] 0807 19:49:34.353480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:34.353864 (+ 384us) service_pool.cc:224] Handling call
0807 19:49:37.421516 (+3067652us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:36.149153 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:36.212669 (+ 63516us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:36.212726 (+ 57us) write_op.cc:435] Acquired schema lock
0807 19:49:36.212746 (+ 20us) tablet.cc:662] Decoding operations
0807 19:49:36.214582 (+ 1836us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:36.214720 (+ 138us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:36.214749 (+ 29us) tablet.cc:685] Acquiring locks for 23 operations
0807 19:49:36.349200 (+134451us) tablet.cc:701] Row locks acquired
0807 19:49:36.349224 (+ 24us) write_op.cc:260] PREPARE: finished
0807 19:49:36.349425 (+ 201us) write_op.cc:270] Start()
0807 19:49:36.349591 (+ 166us) write_op.cc:276] Timestamp: P: 1786132176349362 usec, L: 0
0807 19:49:36.349612 (+ 21us) op_driver.cc:348] REPLICATION: starting
0807 19:49:36.350450 (+ 838us) log.cc:844] Serialized 9259 byte log entry
0807 19:49:36.396609 (+ 46159us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f68784e0a5c 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb
0807 19:49:36.609749 (+213140us) op_driver.cc:464] REPLICATION: finished
0807 19:49:36.782151 (+172402us) write_op.cc:301] APPLY: starting
0807 19:49:36.782266 (+ 115us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:36.797783 (+ 15517us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:36.797819 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:36.806265 (+ 8446us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:36.812185 (+ 5920us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0807 19:49:36.812236 (+ 51us) write_op.cc:312] APPLY: finished
0807 19:49:36.812658 (+ 422us) log.cc:844] Serialized 157 byte log entry
0807 19:49:36.812844 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:36.813052 (+ 208us) write_op.cc:454] Released schema lock
0807 19:49:36.813106 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:36.813485 (+ 379us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:36.813510 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10946,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":166532,"apply.run_cpu_time_us":5034,"apply.run_wall_time_us":35078,"num_ops":23,"prepare.queue_time_us":279288,"prepare.run_cpu_time_us":22431,"prepare.run_wall_time_us":327431,"raft.queue_time_us":119920,"raft.run_cpu_time_us":7606,"raft.run_wall_time_us":135577,"replication_time_us":259976,"spinlock_wait_cycles":63909760,"thread_start_us":436251,"threads_started":4,"wal-append.queue_time_us":110253}]]}
I20260807 19:49:37.400738 11608 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:37.441920 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:37.402824 11331 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:37.402091 11107 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.448912 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:37.464932 9015 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:37.450217 8901 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:37.467569 9017 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:37.453166 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:37.469823 8740 leader_election.cc:400] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:37.470225 8740 leader_election.cc:403] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260807 19:49:37.471724 8778 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:37.473778 8968 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:37.371031 11756 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260807 19:49:37.431649 11444 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:37.429327 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:37.400386 11620 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260807 19:49:37.441130 11545 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:37.444404 11641 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:37.444990 11771 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:37.444465 11630 raft_consensus.cc:697] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.446987 11331 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:37.481607 11664 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:37.479614 11741 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:37.520376 11660 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:37.475716 10988 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.046s
I20260807 19:49:37.522261 11331 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:37.483466 11760 consensus_queue.cc:1048] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260807 19:49:37.520273 11641 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.537060 11695 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:37.501151 11503 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:37.549897 9015 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:37.546778 11660 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:37.556329 11107 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:37.559085 11777 log.cc:927] Time spent T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.087s user 0.001s sys 0.000s
I20260807 19:49:37.559285 11720 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.492928 8904 raft_consensus.cc:2376] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7. Re-sending same reply.
I20260807 19:49:37.494659 11118 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.537308 11771 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:37.512753 11628 consensus_queue.cc:1048] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260807 19:49:37.521257 11608 consensus_queue.cc:1059] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 19:49:37.560449 11486 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:37.581761 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 108090us
I20260807 19:49:37.575393 11660 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:37.605086 11768 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260807 19:49:37.609511 11465 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:37.641325 11465 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:37.527989 11055 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.614384 11641 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.572896 11522 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:37.621025 11118 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.621903 11720 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.574036 11630 consensus_queue.cc:237] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.623942 11486 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.636845 8856 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:37.688627 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 51928.7us
I20260807 19:49:37.615753 11771 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:37.620042 11627 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:37.683008 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 213310us
W20260807 19:49:37.644047 11751 log.cc:927] Time spent T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
I20260807 19:49:37.684813 11545 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:37.677358 11770 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:37.717199 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 238599us
I20260807 19:49:37.649475 11082 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.064s
I20260807 19:49:37.687856 11743 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:37.713835 10988 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:37.723443 11779 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.075s user 0.000s sys 0.001s
I20260807 19:49:37.691495 11461 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:37.708930 11760 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:37.735280 9021 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:37.765516 8903 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:37.738462 8779 raft_consensus.cc:1275] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 19:49:37.739144 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:37.767130 9027 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:37.741127 8778 raft_consensus.cc:1275] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 19:49:37.741443 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:37.759452 8902 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:37.735142 11545 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.732636 11465 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.728116 11770 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:37.909876 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:37.908579 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:37.851372 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:37.836068 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 69184.5us
I20260807 19:49:38.025048 8901 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:37.829919 8911 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:37.739005 11720 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:37.825949 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.030867 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:37.954061 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.047201 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.035506 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:37.939168 8908 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:38.047931 9022 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:37.857272 9027 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:37.758487 11688 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.023710 8913 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:37.899571 8903 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:38.050658 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:38.052613 8915 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:37.758953 11713 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.051602 9022 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:38.050594 11695 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:37.737829 11743 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.051918 11720 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:38.055909 11804 log.cc:927] Time spent T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260807 19:49:38.056882 11720 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.923131 11745 raft_consensus.cc:3055] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:38.057634 11745 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:37.736511 11503 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:38.058346 11811 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260807 19:49:38.058710 11745 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.058959 11503 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.060038 11745 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:38.061733 11545 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:37.866832 11790 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:38.062922 8915 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:38.063189 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.067942 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:38.068257 9017 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:37.765086 11765 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:37.980952 11524 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:38.057535 11801 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260807 19:49:37.979981 11798 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.095067 11798 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.095484 8778 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260807 19:49:37.980141 11792 log.cc:927] Time spent T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.212s user 0.000s sys 0.001s
W20260807 19:49:37.866483 11800 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.140s user 0.000s sys 0.001s
I20260807 19:49:37.739566 10988 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.071539 8901 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:38.045418 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:38.073195 9015 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:38.076148 9019 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:37.914513 8904 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:38.065173 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:38.079392 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 125453us
I20260807 19:49:38.129868 8849 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:38.080302 9027 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:37.820070 11760 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:37.862679 11797 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.081010 11790 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:38.085820 11720 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.143194 8849 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.147583 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.148384 11515 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:38.092840 11641 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.094437 11770 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.056641 11743 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:37.760481 11461 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.099275 8776 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:38.100873 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:37.836404 11620 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.243s
W20260807 19:49:38.118345 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:38.122978 9021 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:38.127655 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:37.890427 11660 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.125402 8856 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:38.177323 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 52030.5us
I20260807 19:49:38.143898 11798 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.144469 11623 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:38.145139 11608 raft_consensus.cc:2764] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260807 19:49:38.150915 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.101809 11524 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.174063 9015 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:38.203716 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.071163 11713 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.143280 11688 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.184239 8902 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260807 19:49:38.154966 11515 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.154125 8913 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:38.190577 8849 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.195504 9027 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:38.196233 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.224596 9021 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:38.194464 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.198185 8906 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:38.201725 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.206588 8776 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260807 19:49:38.227803 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 117366us
I20260807 19:49:38.182596 11486 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.187320 11743 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:38.085127 11765 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:38.233014 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 167920us
W20260807 19:49:38.247089 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:38.236760 8911 raft_consensus.cc:2410] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:38.251127 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.207038 11657 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.160907 11641 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.215322 11798 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.222668 11065 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:38.257633 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 64483.4us
I20260807 19:49:38.262410 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.264482 11539 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:38.265509 11713 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.254690 9008 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260807 19:49:38.256726 11743 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:38.268651 11743 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.154631 11797 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.276321 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:38.256305 11444 consensus_meta.cc:287] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.761s user 0.012s sys 0.000s
I20260807 19:49:38.254236 11118 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.245119 11812 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:38.231388 11466 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260807 19:49:38.231144 11628 consensus_queue.cc:1048] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260807 19:49:38.257313 11515 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:38.252813 9015 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
W20260807 19:49:38.294600 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:38.309113 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 45327.9us
I20260807 19:49:38.227985 11688 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:38.312567 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 73993.2us
I20260807 19:49:38.260494 11065 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.260577 11810 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260807 19:49:38.260885 11798 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.311441 11107 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:38.325402 11783 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.325634 10988 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.325803 11628 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.288004 11118 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:38.284334 11444 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.274389 11486 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260807 19:49:38.323196 11770 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:38.346467 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:38.278535 11627 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260807 19:49:38.311398 11759 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.329324 11824 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:38.363323 9019 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:38.330209 11461 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.351183 11466 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:38.379348 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 155157us
W20260807 19:49:38.381456 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 190981us
I20260807 19:49:38.389132 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.390492 9015 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 19:49:38.412834 9022 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:38.415248 8780 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:38.410706 9020 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
W20260807 19:49:38.387847 11331 consensus_meta.cc:287] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.819s user 0.012s sys 0.001s
I20260807 19:49:38.418530 9023 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:38.415918 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.421232 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.420643 9027 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:38.423561 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:38.367264 11713 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260807 19:49:38.363979 11628 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.426275 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:38.491298 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 65309.3us
W20260807 19:49:38.429641 11815 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260807 19:49:38.499511 8856 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:38.410706 11660 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.431655 11839 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 19:49:38.309168 11812 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:38.414251 11055 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:38.277184 11760 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.422745 11539 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 19:49:38.490484 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 64509.3us
I20260807 19:49:38.386296 11759 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:38.420642 11524 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:38.499667 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 73477.4us
I20260807 19:49:38.392148 11770 consensus_queue.cc:1059] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260807 19:49:38.500530 10988 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.425697 11461 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.491021 11720 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:38.505664 11515 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:38.510025 11827 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.513819 11628 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.516544 11118 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:38.517014 11641 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:38.517357 11834 log.cc:927] Time spent T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260807 19:49:38.518302 11065 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:38.519102 11816 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.217s user 0.000s sys 0.001s
I20260807 19:49:38.525256 11765 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.419955 11466 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.360241 11783 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.425416 11331 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:38.554466 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 208120us
I20260807 19:49:38.771955 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:38.559213 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:38.773433 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 214283us
I20260807 19:49:38.561033 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:38.776789 9015 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:38.493571 11761 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.203s
I20260807 19:49:38.540172 11465 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:38.581991 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:38.503458 11605 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260807 19:49:38.551141 11539 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:38.714303 11759 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:38.566434 11770 consensus_queue.cc:1243] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer 671d8754170646eaac5748745eba58b4 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260807 19:49:38.715412 11118 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:38.670949 11515 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:38.841665 9018 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:38.762429 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.763216 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.782444 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.711323 11503 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:38.758699 11790 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:38.786301 11821 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.187s user 0.001s sys 0.000s
I20260807 19:49:38.788421 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.789181 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:38.789888 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:38.790382 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:38.675912 11720 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.879001 11850 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:38.879714 11850 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.881557 11466 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:38.882483 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 110598us
I20260807 19:49:38.883272 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:38.844332 8779 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:38.669623 10988 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:38.784180 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:38.770048 11688 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.791376 11839 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.778631 11630 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:38.765985 11797 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:38.876247 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 93972.2us
I20260807 19:49:38.892499 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.893857 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:38.851015 11515 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:38.795420 11771 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260807 19:49:38.896855 8906 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:38.873211 11641 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:38.872166 11444 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.901698 9850 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:38.794024 11657 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260807 19:49:38.793535 9889 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.032s sys 0.057s
I20260807 19:49:38.802631 11465 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.885390 11605 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:38.882265 11850 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.888787 11790 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:38.853317 11660 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260807 19:49:38.791260 11765 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.915831 11798 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.918196 11761 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:38.796154 11539 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:38.965229 11783 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.965992 10988 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:38.934057 11641 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:38.967584 11641 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.885394 8775 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:38.872627 11503 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:38.910509 9889 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s user 0.033s sys 0.058s
I20260807 19:49:38.908987 11623 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:38.928797 11790 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:38.933732 11771 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.935256 11827 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:38.953125 11855 raft_consensus.cc:3055] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
W20260807 19:49:38.955807 11852 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.059s user 0.000s sys 0.001s
I20260807 19:49:38.936959 11830 consensus_queue.cc:1059] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260807 19:49:38.968423 11812 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.897614 11839 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.895835 11630 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.906767 11118 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:38.982630 9897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:38.993294 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:39.002419 9893 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:38.950968 11605 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:38.959488 11761 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.003461 11761 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:38.981072 11790 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.006335 9898 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 19:49:38.958628 11765 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.008257 11835 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:39.010602 11465 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.985781 11771 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.013964 11503 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:38.986513 11855 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:38.993112 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.040766 9008 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
W20260807 19:49:39.041468 11822 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.715s user 0.000s sys 0.010s
I20260807 19:49:39.041210 11855 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.042833 11855 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:39.016316 11833 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.299s user 0.002s sys 0.000s
I20260807 19:49:39.016525 10988 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.016335 11118 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.044914 10988 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.017493 11720 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.019394 11331 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.022152 11630 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.024037 11839 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:38.995020 11827 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.007222 11622 raft_consensus.cc:3055] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:39.053608 11622 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:39.054638 11055 consensus_meta.cc:287] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.515s user 0.009s sys 0.001s
I20260807 19:49:39.054534 11622 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:39.049348 11745 consensus_meta.cc:287] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.989s user 0.009s sys 0.005s
I20260807 19:49:39.003473 11605 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:39.012130 11853 log.cc:927] Time spent T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Append to log took a long time: real 0.267s user 0.002s sys 0.000s
W20260807 19:49:39.029281 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 135511us
I20260807 19:49:39.058159 8849 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.059369 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.039927 11771 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.069519 11720 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:39.071220 11817 raft_consensus.cc:3055] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:39.071727 11817 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:39.053185 11827 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.071929 11720 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:39.073969 9023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1235) took 1110 ms. Trace:
I20260807 19:49:39.053015 11107 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:39.063872 11877 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:49:39.064502 11623 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:39.063862 11622 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:39.065475 11605 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.066756 11515 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.068269 11745 consensus_queue.cc:1048] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:39.074417 9023 rpcz_store.cc:276] 0807 19:49:37.963661 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:37.965267 (+ 1606us) service_pool.cc:224] Handling call
0807 19:49:37.990830 (+ 25563us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:38.420843 (+430013us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:39.073632 (+652789us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54002b89d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:39.073886 (+ 254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937765248}
I20260807 19:49:39.088311 11515 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.101475 8972 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.074366 10988 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.084504 11107 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:39.108062 11082 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260807 19:49:39.114728 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.116742 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.116194 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.116683 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.118927 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.117579 8780 raft_consensus.cc:2368] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:39.120770 8788 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:39.121598 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.122068 8788 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:39.128929 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.109053 11465 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:39.137522 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.109100 11877 raft_consensus.cc:697] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:39.107695 11839 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260807 19:49:39.115142 11605 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:39.142076 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:39.132305 11798 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:39.145294 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:39.157325 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 132758us
I20260807 19:49:39.161796 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.162464 8787 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:39.167599 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.142581 11641 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:39.190222 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.199482 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.196902 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.200042 8901 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.200270 8903 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.201025 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:39.201171 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.159595 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.202265 8903 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.168331 8910 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.201470 8901 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.168692 11461 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:39.143203 11465 raft_consensus.cc:697] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:39.206362 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.208300 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.117956 11817 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.208817 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.209491 8903 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.210372 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.168527 11760 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:39.211036 8903 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.211352 11760 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.211933 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.205946 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.212972 8908 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:39.213135 8903 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.213908 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:39.214680 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.204434 11641 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.215699 8903 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.216048 11839 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.216873 11839 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.217422 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.219146 11839 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.199036 11812 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:39.219425 8903 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.208097 8913 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.220808 11812 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.165186 11743 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.243960 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.244089 8908 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.245210 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.246134 8903 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:39.223906 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.225349 11812 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.227072 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.233186 11461 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.247184 8910 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.249213 11461 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:39.233892 11880 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
I20260807 19:49:39.250008 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.238533 11641 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.240219 11790 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.251221 8787 raft_consensus.cc:2368] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:49:39.251323 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.232815 11877 consensus_queue.cc:237] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.254494 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.255358 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.255951 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:39.256594 9018 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.246055 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.257525 8779 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.256671 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.258320 9027 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:39.257496 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.259068 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.259403 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.260334 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.262683 8901 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:39.258548 8856 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.231825 11539 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.213644 11760 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:39.274688 8789 raft_consensus.cc:2368] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:39.222695 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.161481 11605 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.261828 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.277280 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.263564 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.273284 11461 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:39.282884 9017 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:39.269439 8968 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.277865 8783 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:39.281867 11765 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.164430 11798 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.285321 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:39.300405 8783 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.285328 8968 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.293035 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.293589 8904 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.294473 11790 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.294734 11797 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:39.297444 11839 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:39.313087 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:39.283113 11461 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.298122 11810 consensus_queue.cc:1048] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 19:49:39.144410 11761 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.327874 8908 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:39.299271 9018 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:39.267181 11741 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.329988 11741 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.298491 11605 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:39.312458 9023 raft_consensus.cc:1275] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:39.318764 9022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 959) took 1251 ms. Trace:
I20260807 19:49:39.320524 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.323220 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:39.324716 8968 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.328018 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 977) took 1288 ms. Trace:
I20260807 19:49:39.334985 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.335897 8910 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Handling vote request from an unknown peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:39.331347 9022 rpcz_store.cc:276] 0807 19:49:38.067761 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:38.077623 (+ 9862us) service_pool.cc:224] Handling call
0807 19:49:38.077959 (+ 336us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:38.415346 (+337387us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:39.318447 (+903101us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540043c6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:39.318688 (+ 241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720394624}
I20260807 19:49:39.335680 8911 rpcz_store.cc:276] 0807 19:49:38.028463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:38.045720 (+ 17257us) service_pool.cc:224] Handling call
0807 19:49:39.316186 (+1270466us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001ed2d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:39.316332 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384365312}
I20260807 19:49:39.342613 8910 raft_consensus.cc:2393] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:39.342631 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.344805 8789 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.345866 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.259512 11743 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.348488 8910 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:39.349506 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:39.353709 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.354784 8968 leader_election.cc:400] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:39.358045 8968 leader_election.cc:403] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 19:49:39.312717 11839 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.312433 11797 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:39.322940 11461 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:39.316311 11331 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:39.383134 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.383867 8972 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:39.367442 9897 raft_consensus.cc:2393] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:39.386180 9027 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:39.367259 8903 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:39.315963 11629 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:39.367215 8787 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:39.360613 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.388841 8908 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:39.390162 8908 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260807 19:49:39.390969 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.392660 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.391369 11824 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:39.330762 11539 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.396015 11824 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.380838 11331 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.400707 11850 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.401078 11331 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.401274 11850 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.405251 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.399708 11629 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.407657 11761 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.408824 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:39.409512 9898 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:39.410655 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:39.411263 9897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:39.408057 11817 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.339452 11457 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:39.414726 11457 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.415915 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:39.416155 9897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:39.418180 11457 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.418037 11771 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:39.411350 9893 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:39.424625 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:39.354833 11620 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:39.427392 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.428447 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.428143 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 67) took 4155 ms. Trace:
W20260807 19:49:39.429685 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 45911.3us
I20260807 19:49:39.431767 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.436944 8910 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260807 19:49:39.438454 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.404021 11850 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:39.441028 8910 raft_consensus.cc:2393] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:39.448290 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:39.449070 8910 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.450132 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.373571 11839 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:39.461560 8970 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:39.449226 8903 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260807 19:49:39.463179 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.366431 11524 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.367061 11486 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:39.457885 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.459128 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.460009 8910 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:39.462036 9015 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:39.475224 9015 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:39.473663 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.462615 9021 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:39.373181 11545 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.466521 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.465548 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.477538 8906 raft_consensus.cc:2410] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:49:39.473168 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.477924 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.481613 8910 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:39.483155 8910 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260807 19:49:39.484661 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51784.2us
I20260807 19:49:39.484925 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.492918 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.493963 8911 raft_consensus.cc:2393] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:39.442226 11055 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.442467 11620 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.455062 11756 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:39.506713 8972 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:39.461565 10988 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:39.429615 8629 rpcz_store.cc:276] 0807 19:49:35.272834 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:35.273120 (+ 286us) service_pool.cc:224] Handling call
0807 19:49:37.625545 (+2352425us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b540011a728. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687ef8db41 00007f687f017d8f 00007f687f0194f1 00007f6881532ad1 00007f688153332f 00007f68813f3f0b 00007f68814fa4be 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc
0807 19:49:39.427874 (+1802329us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:37.913278 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:37.917205 (+ 3927us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:37.917251 (+ 46us) write_op.cc:435] Acquired schema lock
0807 19:49:37.917269 (+ 18us) tablet.cc:662] Decoding operations
0807 19:49:38.059796 (+142527us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:38.068816 (+ 9020us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:38.075270 (+ 6454us) tablet.cc:685] Acquiring locks for 5 operations
0807 19:49:38.186703 (+111433us) tablet.cc:701] Row locks acquired
0807 19:49:38.186727 (+ 24us) write_op.cc:260] PREPARE: finished
0807 19:49:38.206415 (+ 19688us) write_op.cc:270] Start()
0807 19:49:38.223619 (+ 17204us) write_op.cc:276] Timestamp: P: 1786132178197083 usec, L: 0
0807 19:49:38.228164 (+ 4545us) op_driver.cc:348] REPLICATION: starting
0807 19:49:38.306558 (+ 78394us) log.cc:844] Serialized 2177 byte log entry
0807 19:49:39.210525 (+903967us) op_driver.cc:464] REPLICATION: finished
0807 19:49:39.329619 (+119094us) write_op.cc:301] APPLY: starting
0807 19:49:39.329713 (+ 94us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:39.329878 (+ 165us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:39.330009 (+ 131us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:39.372639 (+ 42630us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:39.412561 (+ 39922us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0807 19:49:39.412634 (+ 73us) write_op.cc:312] APPLY: finished
0807 19:49:39.413017 (+ 383us) log.cc:844] Serialized 46 byte log entry
0807 19:49:39.413209 (+ 192us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:39.413348 (+ 139us) write_op.cc:454] Released schema lock
0807 19:49:39.413392 (+ 44us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:39.413937 (+ 545us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:39.414008 (+ 71us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":340762,"reported_tablets":11,"spinlock_wait_cycles":20929792,"child_traces":[["op",{"apply.queue_time_us":110655,"apply.run_cpu_time_us":3061,"apply.run_wall_time_us":84220,"num_ops":5,"prepare.queue_time_us":275934,"prepare.run_cpu_time_us":15873,"prepare.run_wall_time_us":648777,"raft.queue_time_us":143509,"raft.run_cpu_time_us":8418,"raft.run_wall_time_us":56765,"replication_time_us":977769,"thread_start_us":411953,"threads_started":4,"wal-append.queue_time_us":253894}]]}
I20260807 19:49:39.326838 11465 consensus_queue.cc:237] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.427814 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.418866 11539 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260807 19:49:39.461311 11641 raft_consensus.cc:3055] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:39.417639 11823 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260807 19:49:39.515334 9850 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:39.510442 11756 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.522934 8783 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:39.590317 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.610195 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.595433 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:39.597127 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 87436.9us
I20260807 19:49:39.501991 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:39.473805 11771 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:39.509766 11620 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:39.525612 10988 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:39.624541 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 198060us
I20260807 19:49:39.527806 11629 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.468638 11881 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260807 19:49:39.495957 11486 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.496840 11524 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.500840 11545 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.611228 11790 raft_consensus.cc:2764] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260807 19:49:39.614777 11082 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:39.660924 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.611913 11641 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:39.615408 11756 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:39.637102 11389 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 19:49:39.661784 11107 consensus_meta.cc:287] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.544s user 0.007s sys 0.003s
I20260807 19:49:39.670050 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.641026 10988 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:39.668373 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 181720us
I20260807 19:49:39.650482 11783 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:39.694401 11783 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.694808 11783 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:39.683576 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.673666 8904 raft_consensus.cc:2393] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:39.685001 9027 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:39.691956 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.683763 11812 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:39.693395 11903 log.cc:927] Time spent T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
W20260807 19:49:39.663621 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 212520us
I20260807 19:49:39.697347 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.708425 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:39.692241 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 47588us
I20260807 19:49:39.699280 8787 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:39.614379 11867 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260807 19:49:39.662431 11824 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:39.668520 11082 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.709556 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.622558 11466 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260807 19:49:39.675979 11765 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:39.693662 11760 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.648095 11823 consensus_queue.cc:1048] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260807 19:49:39.647873 11807 consensus_queue.cc:1059] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260807 19:49:39.715456 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.710660 11812 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.724871 11522 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 19:49:39.725584 11896 log.cc:927] Time spent T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.088s user 0.001s sys 0.000s
I20260807 19:49:39.727372 11524 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.727842 11641 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.704640 11798 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.700114 11389 raft_consensus.cc:697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:39.733173 11630 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:39.660075 11827 consensus_queue.cc:1059] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260807 19:49:39.684832 11771 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:39.717365 11118 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260807 19:49:39.748649 11622 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:39.701890 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:39.798141 8906 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:39.769630 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 54255.7us
I20260807 19:49:39.747607 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:39.799168 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.799924 8776 raft_consensus.cc:2410] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:39.750226 11539 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260807 19:49:39.712112 11741 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.698213 11107 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.746659 11082 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:39.771337 11745 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:39.812111 11906 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.105s user 0.000s sys 0.001s
I20260807 19:49:39.780522 11461 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.856945 11461 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.786252 11743 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.788303 11824 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:39.858870 11824 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:39.860069 11824 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.861634 11824 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:39.860524 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 345276us
I20260807 19:49:39.868607 11882 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:39.869275 11882 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.868062 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:39.876685 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:39.877517 8904 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:39.888118 8778 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:39.873376 11882 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.896051 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:39.906263 11899 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.158s user 0.001s sys 0.001s
I20260807 19:49:39.911374 11882 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:39.928297 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 421678us
W20260807 19:49:39.935918 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 151239us
I20260807 19:49:39.937666 11926 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:39.938340 11926 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.940660 11926 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.918270 11486 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:39.944949 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:39.931531 11895 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.245s user 0.000s sys 0.002s
I20260807 19:49:39.896750 11389 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:39.943969 11465 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:39.954555 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:39.955102 11465 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.955541 11812 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:39.957281 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:39.958508 8789 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s user 0.014s sys 0.018s
W20260807 19:49:39.958943 8789 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s user 0.014s sys 0.018s
I20260807 19:49:39.965662 11465 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.918313 11771 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:39.970664 11465 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:39.971174 11465 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:39.971534 11465 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:39.972901 10988 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:39.977101 11910 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.094s user 0.000s sys 0.001s
I20260807 19:49:39.975189 10988 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:39.955184 11444 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:39.979656 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:39.986564 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:39.977885 11855 consensus_meta.cc:287] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.935s user 0.012s sys 0.005s
I20260807 19:49:39.988233 8906 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:39.961109 11745 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:39.991811 11817 consensus_meta.cc:287] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.574s user 0.009s sys 0.004s
I20260807 19:49:39.998816 8972 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.004215 8788 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
W20260807 19:49:40.001543 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 227280us
I20260807 19:49:39.984697 11444 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.004722 8788 pending_rounds.cc:85] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:39.993867 11798 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:39.992960 11107 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.002934 11627 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:40.008422 8788 pending_rounds.cc:107] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 19:49:40.015945 8788 raft_consensus.cc:2889] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:40.009330 11444 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.009173 11915 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:40.006791 8972 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.016891 8856 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:40.010447 11798 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.026772 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:40.013207 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 300047us
W20260807 19:49:40.023030 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:40.018570 11055 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:40.030113 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.035597 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.038218 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.038561 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.038945 8783 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:40.039140 8787 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
W20260807 19:49:40.040441 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 241364us
I20260807 19:49:40.041419 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.035565 8856 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:40.027580 11915 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.052609 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.052196 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.029366 11798 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.016870 11627 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.053984 11461 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:40.059851 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:40.050441 11331 consensus_meta.cc:287] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.632s user 0.012s sys 0.000s
I20260807 19:49:40.059852 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:40.073719 8787 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.057578 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.032976 11929 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:40.077268 11929 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.077708 11929 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.045549 11522 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:40.060911 11627 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:40.080230 11522 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.080721 11522 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.034258 11082 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:40.085899 11331 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.072578 11461 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.088394 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:40.088965 11461 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.089648 9008 raft_consensus.cc:2368] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:40.092730 11627 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.042296 11466 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:40.094565 11466 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.095628 11798 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.099948 11929 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.102456 11935 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:40.102849 11930 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:40.109532 11082 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:40.113580 11942 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.114243 11942 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.115267 11756 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.116617 9018 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:40.118876 11783 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.056164 11915 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.120193 9020 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:40.044289 10988 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:40.063479 11931 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:40.120997 10988 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:40.121280 11931 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:40.121721 11931 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:40.032713 11827 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:40.063261 11465 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.034252 11855 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:40.075798 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.076555 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.054523 11720 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.119201 11942 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.077643 9022 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:49:40.135332 11797 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.136329 11055 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.136757 11797 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.136965 11055 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:40.143429 11904 log.cc:927] Time spent T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.479s user 0.002s sys 0.000s
I20260807 19:49:40.146286 11465 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.149444 11055 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.076366 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:40.069087 8779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s user 0.004s sys 0.010s
I20260807 19:49:40.131470 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:40.160588 8779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s user 0.004s sys 0.010s
I20260807 19:49:40.164147 8787 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:40.165714 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:40.158942 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 82694.8us
I20260807 19:49:40.133358 11082 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.121976 10988 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.188117 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.192190 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:40.192651 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.193130 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.193656 8901 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:40.193782 8913 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:40.194864 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.195773 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.196189 8972 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.197599 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.199981 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:40.182133 11620 consensus_meta.cc:287] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.527s user 0.009s sys 0.002s
I20260807 19:49:40.169471 11790 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:40.199970 11935 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:40.208500 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.224604 8913 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.226577 11522 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.152276 11915 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:40.231096 11915 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:40.233158 11855 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:40.203536 11942 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.234472 11942 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.237977 11783 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.238629 11824 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:40.239054 11824 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.239413 11824 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.202277 11082 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.244540 8910 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:40.245544 11620 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.247848 11620 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.248667 9019 raft_consensus.cc:2368] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4.
I20260807 19:49:40.252910 11444 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.209100 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.253829 11664 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260807 19:49:40.254360 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 45333.9us
I20260807 19:49:40.254815 11107 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:40.255211 11444 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.255522 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.256657 11935 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:40.257098 8740 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.215653 11756 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.216539 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.200774 11629 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:40.259172 11942 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.259048 11629 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.244270 11082 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:40.261293 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:40.261641 11629 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.219901 11817 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:40.222121 11760 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:40.188714 11827 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:40.257347 11790 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 19:49:40.264323 11827 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.266685 11771 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:40.189950 11840 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:49:40.223678 11839 consensus_meta.cc:287] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.747s user 0.006s sys 0.006s
I20260807 19:49:40.253435 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1438) took 1013 ms. Trace:
I20260807 19:49:40.268491 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:40.209026 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.293282 8911 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.277321 8901 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.287354 8913 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:40.290894 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:40.292846 8780 rpcz_store.cc:276] 0807 19:49:39.239627 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.241349 (+ 1722us) service_pool.cc:224] Handling call
0807 19:49:39.241816 (+ 467us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:49:39.242215 (+ 399us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:39.242242 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:49:39.256125 (+ 13883us) log.cc:844] Serialized 52 byte log entry
0807 19:49:39.429096 (+172971us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:39.429141 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:39.429218 (+ 77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:40.142761 (+713543us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:40.253081 (+110320us) raft_consensus.cc:1685] finished
0807 19:49:40.253103 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:40.253347 (+ 244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":16707328,"thread_start_us":114482,"threads_started":1,"wal-append.queue_time_us":231459}
I20260807 19:49:40.210142 11720 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:40.257669 11524 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:40.254555 11664 consensus_queue.cc:1243] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer 671d8754170646eaac5748745eba58b4 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260807 19:49:40.315498 8901 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:40.291090 11840 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:40.255571 11107 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.057s
I20260807 19:49:40.299683 11935 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.321782 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.310637 11524 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:40.336193 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:40.334602 8911 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:40.316093 11850 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:40.344089 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.345144 8906 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.345181 8849 leader_election.cc:400] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:49:40.345890 8849 leader_election.cc:403] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:40.343742 11930 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:40.300000 11839 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.293393 11928 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260807 19:49:40.332706 11840 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:40.356222 11912 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.477s user 0.001s sys 0.003s
I20260807 19:49:40.353863 11930 raft_consensus.cc:697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:40.341256 11524 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.365980 8906 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:40.312115 11608 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.370280 9838 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:40.337683 11664 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:40.385455 11505 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:40.385778 11622 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:40.390202 11608 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.397773 11505 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:40.398320 11505 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:40.398955 11608 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.400907 11461 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.402424 8908 raft_consensus.cc:2393] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:40.404706 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.407210 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.410405 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.416901 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.359443 9008 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 19:49:40.418246 8783 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:40.425352 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.419713 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:40.421003 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:40.355620 11522 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.421263 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.430261 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.431332 8788 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:49:40.431716 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 61533.3us
I20260807 19:49:40.433744 8788 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:40.386639 11941 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
W20260807 19:49:40.435304 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 233282us
I20260807 19:49:40.433183 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.439473 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.443912 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.444679 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.403481 11461 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.445268 8913 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:40.445516 8903 raft_consensus.cc:2410] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:40.445921 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.447225 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.417281 11539 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:40.453159 8788 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:40.454070 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:40.454227 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.452322 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.452277 8911 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.452766 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.456404 8910 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.341876 11935 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:40.485234 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:40.535451 9015 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:40.538993 9015 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:40.541085 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:40.545373 8910 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:40.546927 8913 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:40.547113 8911 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:40.551149 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:40.551751 9022 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.552731 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.553194 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1287) took 1385 ms. Trace:
I20260807 19:49:40.553399 8911 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.558666 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.553603 8902 rpcz_store.cc:276] 0807 19:49:39.167501 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.167642 (+ 141us) service_pool.cc:224] Handling call
0807 19:49:39.167991 (+ 349us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:39.443327 (+275336us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:39.443417 (+ 90us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:39.445618 (+ 2201us) log.cc:844] Serialized 26 byte log entry
0807 19:49:39.634900 (+189282us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:39.635042 (+ 142us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:39.735124 (+100082us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003d7cd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:39.735185 (+ 61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:40.352961 (+617776us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:40.552986 (+200025us) raft_consensus.cc:1685] finished
0807 19:49:40.553010 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:40.553137 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10372,"raft.queue_time_us":8624,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":15033,"spinlock_wait_cycles":618291200,"thread_start_us":148528,"threads_started":1,"wal-append.queue_time_us":315342}
I20260807 19:49:40.564692 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:40.571807 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 137611us
I20260807 19:49:40.572935 9838 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:40.579448 8911 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:40.577489 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.556942 11850 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:40.542657 11055 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.554838 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.532064 11539 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.590968 8972 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:40.591491 9023 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:40.577474 11389 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:40.599920 9023 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260807 19:49:40.600247 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 146214us
I20260807 19:49:40.600862 9022 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:40.601210 8740 leader_election.cc:400] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:40.601543 8740 leader_election.cc:403] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:40.605803 8779 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:40.607995 9023 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:40.610761 9023 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:40.612874 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:40.596500 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:40.617772 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:40.610432 11950 log.cc:927] Time spent T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.271s user 0.000s sys 0.001s
I20260807 19:49:40.595530 11914 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:40.620561 11933 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.543s user 0.005s sys 0.001s
I20260807 19:49:40.590273 11055 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.604986 11824 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.618889 11966 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:40.642345 11539 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.615559 8972 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:40.621711 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:40.610019 11629 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:40.662266 9018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 978) took 1204 ms. Trace:
I20260807 19:49:40.636193 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:40.679560 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 43481.7us
I20260807 19:49:40.640731 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.642197 8906 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:40.682376 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.683218 8906 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:40.642463 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 972) took 1188 ms. Trace:
I20260807 19:49:40.643714 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.644085 11827 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:40.678684 9018 rpcz_store.cc:276] 0807 19:49:39.458099 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.473374 (+ 15275us) service_pool.cc:224] Handling call
0807 19:49:40.662055 (+1188681us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b540002d854. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:40.662197 (+ 142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472088832}
I20260807 19:49:40.645113 8913 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:40.645712 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.688621 8906 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260807 19:49:40.684473 9019 rpcz_store.cc:276] 0807 19:49:39.456008 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.456310 (+ 302us) service_pool.cc:224] Handling call
0807 19:49:40.642250 (+1185940us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400069ad4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:40.642397 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1801693952}
I20260807 19:49:40.625442 11839 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:40.664981 11953 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.304s user 0.001s sys 0.001s
I20260807 19:49:40.649644 8903 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:40.702394 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:40.619078 8904 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:40.703926 11466 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:40.663084 11931 consensus_meta.cc:287] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.541s user 0.010s sys 0.002s
I20260807 19:49:40.657207 11545 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.605471 11957 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260807 19:49:40.581122 11930 consensus_queue.cc:237] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.655784 8911 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:40.606931 11627 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.673714 11515 raft_consensus.cc:3055] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:40.691888 11827 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.665100 11850 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 5
W20260807 19:49:40.667929 11956 log.cc:927] Time spent T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.084s user 0.000s sys 0.001s
I20260807 19:49:40.668447 11622 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260807 19:49:40.671034 11055 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.673497 11824 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.685961 11914 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:40.692618 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:40.783670 9017 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:40.659421 11935 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.699266 11630 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:40.647061 11524 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.652305 11798 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.706485 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:40.653978 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.664474 9027 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260807 19:49:40.661721 8908 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:40.682379 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:40.677139 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 61646.7us
I20260807 19:49:40.708113 8902 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260807 19:49:40.676627 11797 consensus_meta.cc:287] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.503s user 0.008s sys 0.000s
I20260807 19:49:40.712690 11929 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:40.813520 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 107122us
I20260807 19:49:40.813970 8787 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260807 19:49:40.814430 8787 pending_rounds.cc:85] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:40.814656 8787 pending_rounds.cc:107] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260807 19:49:40.814745 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:40.786084 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:40.712548 11515 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:40.798724 11877 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s user 0.017s sys 0.011s
I20260807 19:49:40.706332 8904 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [1.3-4.5]
I20260807 19:49:40.807587 9020 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:40.807960 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:40.806730 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 56736.8us
I20260807 19:49:40.814930 8787 raft_consensus.cc:2889] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:40.817988 8908 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:49:40.819239 8901 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
W20260807 19:49:40.661068 11905 log.cc:927] Time spent T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.972s user 0.000s sys 0.004s
I20260807 19:49:40.800132 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:40.895695 8915 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:40.899047 8787 pending_rounds.cc:107] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:40.899582 8787 raft_consensus.cc:2889] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 19:49:40.903198 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 95299us
I20260807 19:49:40.904140 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:40.705302 11931 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.705722 11790 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.121s
I20260807 19:49:40.801928 11539 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:40.900483 11877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s user 0.018s sys 0.020s
I20260807 19:49:40.804106 11914 raft_consensus.cc:697] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:40.827195 11486 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:40.828985 11967 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:40.906961 11840 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.509s user 0.010s sys 0.000s
I20260807 19:49:40.893528 11630 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:40.974026 11630 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.902384 11824 consensus_queue.cc:1059] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 19:49:40.911926 11929 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:40.941015 11790 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:40.942265 11524 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.943718 11664 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:40.905901 9008 raft_consensus.cc:1240] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:40.812987 11935 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:40.893169 11771 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:40.954766 8915 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:40.663930 11505 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:40.959969 9021 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:40.924597 11973 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260807 19:49:40.963951 9017 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:40.976063 9020 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:41.006213 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:41.007071 9020 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:40.942660 11956 log.cc:927] Time spent T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.094s user 0.000s sys 0.001s
I20260807 19:49:40.977205 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.008119 9023 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:40.911587 11798 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:41.009073 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.009550 11798 raft_consensus.cc:697] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:40.917639 11817 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:40.917928 11797 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:41.010344 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.010465 11798 consensus_queue.cc:237] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.010809 11797 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:41.011806 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:40.970433 11055 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.014422 11055 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.015059 9020 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:40.950652 11539 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.994637 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:40.981169 11790 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.018378 11539 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:40.892953 11926 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
W20260807 19:49:41.013445 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 92998.2us
I20260807 19:49:40.994920 11967 raft_consensus.cc:3055] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:40.977166 11486 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:40.977029 11840 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:40.991693 11929 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:41.023428 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:40.952622 11515 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.032925 8856 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:41.020229 11926 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:41.019184 9020 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:41.025108 9893 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:41.038785 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.044754 8776 raft_consensus.cc:2393] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:41.045838 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 103826us
I20260807 19:49:41.046509 11461 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.047004 11461 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.045467 8856 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:41.049085 11461 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.049631 11461 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:41.050161 11461 raft_consensus.cc:697] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:41.047773 11930 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.003s
I20260807 19:49:41.045650 11855 consensus_queue.cc:1059] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:41.050952 11877 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.051493 9893 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:41.019410 11055 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:40.990919 11935 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.020256 9889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33452 (request call id 217) took 1308 ms. Trace:
I20260807 19:49:41.052635 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.054435 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.047910 8628 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:41.032512 11797 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:41.059923 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.032359 11967 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.066227 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.062529 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.051369 11824 raft_consensus.cc:3055] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:41.022464 11817 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.053958 11926 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.073225 11824 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.074961 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:41.075556 11974 log.cc:927] Time spent T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260807 19:49:41.076006 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.077360 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.079677 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.084811 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.053522 11982 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:41.065897 11622 raft_consensus.cc:3055] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:41.069711 11515 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:41.087522 11622 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:41.088586 11622 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.089756 11622 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:41.094056 11840 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:41.094206 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:41.040047 11790 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.095376 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.095777 11790 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:41.095989 8903 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:41.096745 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1183) took 1818 ms. Trace:
W20260807 19:49:41.098354 11979 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260807 19:49:41.087491 11641 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:41.082300 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.084945 9018 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:41.103075 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.068264 8628 catalog_manager.cc:5697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:41.097131 8775 rpcz_store.cc:276] 0807 19:49:39.278077 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.295044 (+ 16967us) service_pool.cc:224] Handling call
0807 19:49:39.303066 (+ 8022us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:39.331522 (+ 28456us) raft_consensus.cc:1534] Early marking committed up to index 2
0807 19:49:39.368276 (+ 36754us) log.cc:844] Serialized 14 byte log entry
0807 19:49:39.492784 (+124508us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:39.621997 (+129213us) log.cc:844] Serialized 26 byte log entry
0807 19:49:39.657167 (+ 35170us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:39.689410 (+ 32243us) log.cc:844] Serialized 14 byte log entry
0807 19:49:39.708318 (+ 18908us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:39.745313 (+ 36995us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:40.640172 (+894859us) spinlock_profiling.cc:243] Waited 7.99 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:41.096496 (+456324us) raft_consensus.cc:1685] finished
0807 19:49:41.096525 (+ 29us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:41.096680 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33002,"prepare.queue_time_us":29005,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":118,"raft.queue_time_us":10187,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":10614,"spinlock_wait_cycles":18380032,"thread_start_us":102926,"threads_started":1,"wal-append.queue_time_us":230481}
I20260807 19:49:41.111960 9020 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:41.112588 8790 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:41.114694 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.117404 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.119535 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1439) took 1879 ms. Trace:
I20260807 19:49:41.119922 8778 rpcz_store.cc:276] 0807 19:49:39.239911 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.240544 (+ 633us) service_pool.cc:224] Handling call
0807 19:49:39.240905 (+ 361us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:39.888711 (+647806us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:41.119289 (+1230578us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54000f5854. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:41.119475 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487947520}
I20260807 19:49:41.051494 11877 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:41.122181 8628 catalog_manager.cc:5697] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:41.097605 9008 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:41.128329 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.086254 11982 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:41.101855 11827 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.026152 11914 consensus_queue.cc:237] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.131635 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.138018 8790 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:41.162268 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.164505 8775 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 4
I20260807 19:49:41.126179 8628 catalog_manager.cc:5697] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 4 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:41.167515 8628 catalog_manager.cc:5697] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 4 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:41.170192 8628 catalog_manager.cc:5697] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 2, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 2 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:41.173101 8628 catalog_manager.cc:5697] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 3, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
W20260807 19:49:41.175626 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 122072us
I20260807 19:49:41.120702 11940 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.070331 11461 consensus_queue.cc:237] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.181471 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.128470 11931 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:41.177233 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:41.168846 11982 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:41.187489 11981 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.118s user 0.000s sys 0.001s
W20260807 19:49:41.189056 11985 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.130s user 0.001s sys 0.000s
I20260807 19:49:41.117354 11929 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.123509 11967 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.190636 11967 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:41.185626 11940 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.194465 11486 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.195941 11877 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:41.198025 11935 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.199579 11817 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:41.184746 11976 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.125s user 0.000s sys 0.001s
I20260807 19:49:41.201316 11940 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.206928 11935 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:41.207877 11850 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:41.212595 11983 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:41.213963 11771 consensus_queue.cc:1059] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:41.215333 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.215579 11982 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.216861 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.217296 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.226385 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:41.230487 8903 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
W20260807 19:49:41.234198 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:41.166453 11824 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.165791 11926 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.161901 11941 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:41.265666 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.281391 8787 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:41.274575 11987 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.082s user 0.000s sys 0.001s
I20260807 19:49:41.260583 11664 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.174358 11797 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.183163 8628 catalog_manager.cc:5697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:41.299571 8628 catalog_manager.cc:5697] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 1 to 2, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 2 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:41.278497 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:41.264160 8968 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.289901 8787 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:41.292117 8775 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:41.284741 9008 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:41.294683 11926 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:41.286026 11486 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:41.295513 11941 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:41.296592 11935 consensus_queue.cc:1175] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a38f4d6088cf4c8cb510200ca0555801" responder_term: 5 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 19:49:41.302361 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.302690 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.303056 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:41.374212 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 156980us
W20260807 19:49:41.376530 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 73561.8us
I20260807 19:49:41.377580 8968 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.379400 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.335069 8779 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:41.383862 8737 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.346625 11850 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:41.388222 11741 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:41.388809 11524 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.388273 11957 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:41.307906 11664 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.296087 11331 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:41.393795 11664 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.324940 11797 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:41.356113 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:41.385780 8970 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.325218 11973 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:41.398306 8970 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:41.337488 11771 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.338163 11486 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.367431 11761 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:41.412134 8787 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:41.410763 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:41.405685 8968 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.318586 11931 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.425206 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.399170 11957 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:41.429634 8968 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:41.413091 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.411669 11741 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:41.456818 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.457778 8775 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:41.459424 11941 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:41.461781 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.462416 8903 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Handling vote request from an unknown peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:41.462853 8903 raft_consensus.cc:2368] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5.
I20260807 19:49:41.463863 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.464056 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.464529 8915 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:41.394301 11850 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:41.465147 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 81411us
I20260807 19:49:41.465631 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.465731 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.469756 8906 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:41.413358 11622 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:41.467434 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 92342.6us
I20260807 19:49:41.471680 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.473449 8903 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:41.473953 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.476217 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.476397 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.476215 9850 leader_election.cc:400] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5.
I20260807 19:49:41.477412 9850 leader_election.cc:403] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 19:49:41.407294 11627 consensus_queue.cc:1059] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260807 19:49:41.446261 11929 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.480240 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:41.454845 11915 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:41.483477 8915 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:41.484714 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.488201 8906 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:41.487393 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.490562 8904 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:41.490975 11982 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.491673 9022 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:41.059121 9889 rpcz_store.cc:276] 0807 19:49:39.711696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:39.716748 (+ 5052us) service_pool.cc:224] Handling call
0807 19:49:39.777092 (+ 60344us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:39.811119 (+ 34027us) raft_consensus.cc:1534] Early marking committed up to index 7
0807 19:49:39.854156 (+ 43037us) log.cc:844] Serialized 14 byte log entry
0807 19:49:40.023519 (+169363us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:40.025695 (+ 2176us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:49:40.030358 (+ 4663us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:40.994190 (+963832us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b54000e33d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:40.996286 (+ 2096us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:41.001785 (+ 5499us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":224,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":4606,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":8570,"spinlock_wait_cycles":45413248,"thread_start_us":160884,"threads_started":1,"wal-append.queue_time_us":234291,"wal-append.run_cpu_time_us":1055,"wal-append.run_wall_time_us":1003831}
I20260807 19:49:41.493189 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:41.494153 11627 raft_consensus.cc:3055] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:41.494556 11627 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:41.474735 11741 raft_consensus.cc:697] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:41.478250 11622 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.496292 11741 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:41.497639 11622 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.499943 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:41.500659 9022 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:41.476315 11983 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:41.501027 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:41.490451 11993 log.cc:927] Time spent T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.106s user 0.001s sys 0.000s
I20260807 19:49:41.492249 11771 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:41.509230 11983 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:41.510239 11983 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:41.500814 8775 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:41.467459 11505 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.546964 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:41.513185 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.548983 9015 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:41.514566 8775 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260807 19:49:41.549153 12006 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:41.549924 8775 pending_rounds.cc:85] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:41.550261 8775 pending_rounds.cc:107] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 19:49:41.550520 8775 raft_consensus.cc:2889] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:41.515018 8903 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:41.426275 11761 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:41.519107 11931 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:41.554263 11931 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:41.554842 11761 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.556974 11915 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:41.557973 11915 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.527287 11627 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.545444 11641 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:41.545717 8789 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260807 19:49:41.517964 11817 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:41.575480 11817 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:41.575932 11817 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:41.560635 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:41.492156 11990 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.102s user 0.001s sys 0.000s
I20260807 19:49:41.577062 11627 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:41.570850 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.571648 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.572443 9022 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:41.574055 8789 pending_rounds.cc:85] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 1
I20260807 19:49:41.574919 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.581110 8789 pending_rounds.cc:107] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260807 19:49:41.581444 8789 raft_consensus.cc:2889] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:41.580685 9015 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:41.583426 8906 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:41.582110 12001 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.513832 11882 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5.
I20260807 19:49:41.564424 11967 raft_consensus.cc:3055] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:41.590801 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.582317 11850 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.585551 12004 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260807 19:49:41.603493 12004 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:41.605970 8911 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:41.607244 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:41.603724 11771 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.549000 12005 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260807 19:49:41.608795 11771 raft_consensus.cc:3055] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:41.609165 11771 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.512940 11486 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.593179 11967 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.582563 11797 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:41.599025 11641 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:41.597638 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.627560 12006 consensus_queue.cc:1059] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:41.630005 8902 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:41.630617 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.630709 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.640702 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.641340 8915 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:41.641060 8901 raft_consensus.cc:2368] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:41.642251 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:41.643332 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46448.3us
I20260807 19:49:41.644836 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.651252 11630 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:41.651834 11630 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.630641 11973 raft_consensus.cc:697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:41.655296 11973 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:41.631445 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.657634 8911 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:41.639932 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:41.665863 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.666687 9022 raft_consensus.cc:2368] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:41.663546 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.667801 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.671967 9027 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260807 19:49:41.638134 11790 consensus_meta.cc:287] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.524s user 0.008s sys 0.003s
I20260807 19:49:41.665079 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.672251 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.679814 8780 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:41.680387 9018 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:41.681159 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.682472 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.682619 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.683586 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.684158 11745 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.685623 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.684187 8779 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:41.686798 8778 raft_consensus.cc:2410] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260807 19:49:41.686990 8849 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.688053 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.688478 8910 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:41.689600 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.690812 8780 raft_consensus.cc:2368] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5.
I20260807 19:49:41.694433 8790 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:49:41.588909 12001 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.695674 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.695982 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:41.705662 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.696974 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:41.676965 11641 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.703776 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:41.703792 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.706923 8972 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.670799 11461 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:49:41.664177 11827 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:41.713635 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.717401 8972 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:41.726401 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:41.727250 8788 raft_consensus.cc:2376] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5. Re-sending same reply.
I20260807 19:49:41.694877 11797 raft_consensus.cc:3055] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:41.743095 8849 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.700730 11608 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:41.609982 11771 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.707528 11965 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:41.751130 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 44079.9us
I20260807 19:49:41.753304 9893 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:41.751148 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.715482 12005 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:41.715651 11982 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:41.743959 11987 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.058s user 0.001s sys 0.000s
I20260807 19:49:41.735694 11331 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:41.747980 11745 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.715518 11827 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:41.676252 12027 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260807 19:49:41.742030 11797 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:41.712530 11457 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260807 19:49:41.736258 11957 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:41.751794 11515 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:41.747651 8904 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:41.759541 11942 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:41.773181 12006 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260807 19:49:41.765075 11622 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:41.799378 11771 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:41.800439 12038 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:41.748657 11790 consensus_queue.cc:1059] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
W20260807 19:49:41.767117 11850 consensus_queue.cc:1175] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "671d8754170646eaac5748745eba58b4" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 2 last_received_current_leader { term: 3 index: 3 } }
I20260807 19:49:41.759847 12004 consensus_queue.cc:1059] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260807 19:49:41.804473 11055 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:41.835179 11942 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:41.735606 11967 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:41.844273 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 85350us
I20260807 19:49:41.801098 11827 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:41.766698 11965 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:41.846180 11622 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:41.809037 11745 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.845551 11957 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:41.853164 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:41.854939 11331 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:41.849181 12041 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:41.806327 11641 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:41.919838 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:41.836023 11515 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.858996 11942 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:41.867398 11622 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:41.921355 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 208260us
I20260807 19:49:41.936458 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:41.931452 8737 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:41.868746 11745 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:41.886325 11461 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260807 19:49:41.942263 12024 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:49:42.136605 11926 consensus_meta.cc:287] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.795s user 0.012s sys 0.000s
I20260807 19:49:41.869253 11957 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.187551 8776 raft_consensus.cc:2410] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260807 19:49:41.958722 11982 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.220s
I20260807 19:49:42.157243 12014 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:42.191599 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.201695 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.184919 11935 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
W20260807 19:49:42.194258 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 435265us
I20260807 19:49:41.919600 12041 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.172542 12024 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:42.184434 11965 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:42.206156 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 267230us
W20260807 19:49:42.192710 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 310520us
I20260807 19:49:42.218919 9838 leader_election.cc:400] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5.
W20260807 19:49:42.193240 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 287317us
I20260807 19:49:42.219808 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.220095 9027 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260807 19:49:42.206835 12040 log.cc:927] Time spent T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.287s user 0.001s sys 0.000s
I20260807 19:49:42.194437 11741 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.200367 12004 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.181416 11331 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.217235 12024 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:42.224107 9027 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260807 19:49:42.227980 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.228610 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.230245 8740 leader_election.cc:400] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:42.230552 8740 leader_election.cc:403] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:42.232666 8968 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:42.226241 11741 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.236723 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.225622 11926 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:42.235095 12004 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.254302 12004 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.252121 11926 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.205478 12014 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:42.251505 11745 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.251977 11515 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.256300 11926 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.260109 11926 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.261308 11929 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:42.262216 11929 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:42.263526 12014 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.263363 11929 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.262601 11926 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:42.265511 11926 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:42.280687 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:42.283413 12042 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.362s user 0.000s sys 0.001s
W20260807 19:49:42.270002 12035 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.410s user 0.001s sys 0.000s
I20260807 19:49:42.250644 12044 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:42.249316 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.251439 11790 raft_consensus.cc:3055] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:42.246922 11935 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.269407 12041 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:42.275941 12064 log.cc:927] Time spent T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
W20260807 19:49:42.298496 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.329217 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 397977us
I20260807 19:49:42.333560 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:42.304847 11926 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:42.304924 11817 consensus_meta.cc:287] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.729s user 0.010s sys 0.002s
W20260807 19:49:42.342377 8968 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.078s (SENT) [suppressed 7 similar messages]
I20260807 19:49:42.299595 11790 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:42.341109 11817 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:42.353490 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 133726us
I20260807 19:49:42.354931 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.357070 8849 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:42.359659 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:42.304203 11935 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.357172 11461 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:42.362589 12072 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:49:42.364769 12058 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.074s user 0.001s sys 0.000s
I20260807 19:49:42.364475 12024 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:42.365788 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 139810us
I20260807 19:49:42.375880 8856 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:42.360178 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 123478us
I20260807 19:49:42.368031 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:42.344439 11926 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.377429 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:42.372160 11771 consensus_meta.cc:287] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.525s user 0.011s sys 0.000s
I20260807 19:49:42.371434 11915 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:42.387935 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.401104 8790 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:42.370174 12086 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.371124 12072 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:42.472121 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 93439.8us
I20260807 19:49:42.473263 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:42.470233 11539 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:42.381143 12079 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 19:49:42.471473 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 58077.9us
W20260807 19:49:42.475464 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.410650 12070 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:42.468008 12067 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:42.381430 12055 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:42.499307 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.475701 11745 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:42.473583 11741 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:42.501823 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 122855us
I20260807 19:49:42.479519 12072 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:42.485666 12079 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:42.374800 11790 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:42.501528 11942 consensus_meta.cc:287] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.540s user 0.000s sys 0.010s
I20260807 19:49:42.473843 11771 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260807 19:49:42.525714 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.527686 8775 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:42.529320 12105 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:42.530033 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.527045 8778 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:42.537425 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.538117 8783 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:42.539958 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:42.540908 8778 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:42.541815 8775 raft_consensus.cc:2393] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:42.477033 12086 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:42.532294 11622 consensus_meta.cc:287] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.588s user 0.011s sys 0.000s
W20260807 19:49:42.539166 11827 consensus_meta.cc:287] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.675s user 0.003s sys 0.009s
I20260807 19:49:42.518049 12079 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:42.500037 12091 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260807 19:49:42.550992 8908 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:42.548592 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.556332 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.557072 8775 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:42.558404 8775 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:49:42.513875 11745 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.562480 11745 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.564190 12100 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:42.564723 12100 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:42.565107 12100 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:42.560448 11515 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:42.531477 11641 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260807 19:49:42.575258 8778 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:49:42.588183 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 177097us
I20260807 19:49:42.527685 11942 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:42.592852 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.560614 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:42.597216 12104 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:42.600775 12087 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.185s
I20260807 19:49:42.578454 11817 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.551966 11622 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:42.602859 12095 log.cc:927] Time spent T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.187s user 0.001s sys 0.000s
I20260807 19:49:42.608712 12024 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.624219 12001 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.627192 11486 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:42.554431 11827 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.630959 11827 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.632285 8910 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:42.634698 11942 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.617585 12104 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.637290 9892 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:49:42.637912 12027 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:42.638336 12027 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:42.639215 12027 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.587622 11929 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:42.614568 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 141407us
I20260807 19:49:42.643857 8972 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:42.615222 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 63562.2us
I20260807 19:49:42.588898 11515 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.592528 11641 consensus_queue.cc:1243] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Peer 39975243456d4452b4a4870ac7ebafc5 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260807 19:49:42.622874 11622 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.610783 12086 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.651721 11790 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:42.625612 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1337) took 1150 ms. Trace:
I20260807 19:49:42.652684 8903 rpcz_store.cc:276] 0807 19:49:41.475328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.476151 (+ 823us) service_pool.cc:224] Handling call
0807 19:49:42.625451 (+1149300us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b54002fb054. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:42.625554 (+ 103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85723392}
I20260807 19:49:42.659427 8972 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:42.591764 8856 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:42.641436 11797 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.649466 12104 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.647179 11545 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:49:42.651408 12028 log.cc:927] Time spent T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Append to log took a long time: real 0.877s user 0.011s sys 0.000s
I20260807 19:49:42.659132 11622 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:42.714082 11622 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.714323 11545 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:42.717852 11664 raft_consensus.cc:3055] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:42.718312 11664 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:42.718459 12118 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:42.718897 12118 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:42.719225 11664 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.711524 11741 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260807 19:49:42.708972 11957 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:42.719238 12118 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:42.729671 11797 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:42.678942 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 87260.7us
I20260807 19:49:42.732838 8856 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:42.709921 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.645963 12027 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:42.735186 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.741672 8740 leader_election.cc:400] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:42.743263 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:42.748670 12113 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:42.749480 12113 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:42.714884 11515 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.756256 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.762219 9017 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:42.757467 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:42.759975 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:42.770303 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.756729 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.759440 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.770956 9021 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:49:42.781006 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 121665us
I20260807 19:49:42.781924 8972 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:42.781333 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:42.788594 8910 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260807 19:49:42.788697 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 263860us
I20260807 19:49:42.789501 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:42.790370 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:42.791343 8915 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:42.795557 8902 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:42.752236 11942 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:42.800976 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:42.657655 11929 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.803761 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.810832 9017 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:42.751070 11622 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:42.770923 12038 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.811147 8968 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:42.795243 11850 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:42.815276 8787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1490) took 1608 ms (client timeout 1731 ms). Trace:
W20260807 19:49:42.825551 8787 rpcz_store.cc:269] 0807 19:49:41.207121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.302072 (+ 94951us) service_pool.cc:224] Handling call
0807 19:49:42.815064 (+1512992us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b54000974d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:42.815209 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83472640}
I20260807 19:49:42.826988 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.799985 11524 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:42.793143 11827 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:42.807096 11957 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260807 19:49:42.834102 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.834860 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.847267 8787 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:42.811949 12086 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260807 19:49:42.888125 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:42.889050 8787 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:42.816438 12103 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
W20260807 19:49:42.894129 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 83035.3us
I20260807 19:49:42.894698 11817 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:42.901988 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.906852 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.817447 12110 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.056s
W20260807 19:49:42.912962 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.913858 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.917390 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.831111 12038 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:42.917829 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:42.918759 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 134597us
W20260807 19:49:42.918298 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:42.919937 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:42.841035 11827 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.953336 8787 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:42.921206 12121 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:42.954655 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:42.957741 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:42.959824 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.960628 9022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1011) took 1293 ms. Trace:
I20260807 19:49:42.961072 9022 rpcz_store.cc:276] 0807 19:49:41.668638 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.669096 (+ 458us) service_pool.cc:224] Handling call
0807 19:49:42.957530 (+1288434us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:42.960553 (+ 3023us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17055488}
I20260807 19:49:42.932876 11622 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:42.919076 11817 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.966244 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:42.960003 11942 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:42.972126 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 52244.4us
I20260807 19:49:42.939401 10988 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:42.922801 12079 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:42.939714 11545 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:42.975252 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:42.970007 11957 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260807 19:49:42.985996 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.989492 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:42.990865 8787 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:42.978369 12063 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:42.973904 12114 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:42.973546 11942 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:43.005112 12109 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.189s user 0.001s sys 0.000s
I20260807 19:49:43.019683 8787 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:42.972924 11622 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.028716 8788 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260807 19:49:43.029115 12138 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:43.031563 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1190) took 2618 ms. Trace:
I20260807 19:49:42.976645 12121 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.000795 11929 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.013572 12114 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.076598 12114 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.003528 12063 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:43.035259 12072 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.508s user 0.008s sys 0.001s
I20260807 19:49:43.037779 8789 rpcz_store.cc:276] 0807 19:49:40.413490 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:40.413594 (+ 104us) service_pool.cc:224] Handling call
0807 19:49:40.413869 (+ 275us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:41.581950 (+1168081us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:43.031311 (+1449361us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400328a54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:43.031498 (+ 187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2601933056}
I20260807 19:49:43.084197 8789 pending_rounds.cc:85] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:43.039223 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:43.082726 8968 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:42.970564 12139 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260807 19:49:43.070582 11914 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:43.069998 12138 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.071103 11641 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:43.073596 12146 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
I20260807 19:49:43.084626 8789 pending_rounds.cc:107] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 19:49:43.110703 8789 raft_consensus.cc:2889] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 19:49:43.097937 9898 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33434 (request call id 131) took 1634 ms (client timeout 1613 ms). Trace:
W20260807 19:49:43.112699 9898 rpcz_store.cc:269] 0807 19:49:41.464008 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.464451 (+ 443us) service_pool.cc:224] Handling call
0807 19:49:43.038956 (+1574505us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b5800192f98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:43.097847 (+ 58891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5100416}
I20260807 19:49:43.108234 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1341) took 1622 ms. Trace:
I20260807 19:49:43.114710 8913 rpcz_store.cc:276] 0807 19:49:41.485819 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.485960 (+ 141us) service_pool.cc:224] Handling call
0807 19:49:41.751688 (+265728us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b58001e4b98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:41.751938 (+ 250us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:41.958427 (+206489us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:41.958465 (+ 38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:41.958510 (+ 45us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:41.958540 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:43.107920 (+1149380us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54003615d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:43.107997 (+ 77us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:43.108167 (+ 170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478343680}
I20260807 19:49:43.077587 12063 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:43.122290 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 202438us
I20260807 19:49:43.134521 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:43.134923 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:43.136059 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.121153 11827 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.122413 12114 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.124065 11741 consensus_queue.cc:1059] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.041s
I20260807 19:49:43.166627 12090 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:43.167387 12090 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.158886 11331 consensus_queue.cc:1059] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260807 19:49:43.172539 12157 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:43.173281 12157 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:43.173825 12157 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:43.101061 12100 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.107239 11914 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.118840 11641 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.181290 12108 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:43.181747 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:43.102969 12072 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.183421 12079 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.182919 11331 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:43.186815 12152 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:43.195209 12108 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.177866 12172 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:43.191907 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:43.219609 8778 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:43.173475 11824 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:43.163584 12038 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.175707 12114 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.205006 12138 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:43.206947 11914 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.190191 12118 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.195539 12055 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:43.210844 11641 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.211369 12090 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.175249 11827 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:43.182884 12110 raft_consensus.cc:3055] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
W20260807 19:49:43.213982 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 411139us
I20260807 19:49:43.201608 12123 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.211791 11817 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.211189 12027 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:43.212626 11331 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:43.210958 12121 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:43.244005 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.244277 8856 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:43.246259 12110 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:43.229152 12152 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.259296 12110 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.260463 12110 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:43.261539 11331 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.262966 11331 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:43.220709 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 137245us
I20260807 19:49:43.248780 11641 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.247922 11622 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.274111 12148 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:43.274746 12148 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:43.275131 12148 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:43.275576 12118 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.277799 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:43.256222 11741 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:43.278461 11641 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.279186 11741 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.245082 11827 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.265383 12152 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.240728 12055 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.280210 12118 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:43.284598 12114 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.297399 12072 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:43.281968 11641 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.284152 11741 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.266017 12027 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.300064 8856 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:43.356170 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
W20260807 19:49:43.356788 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 56794.8us
I20260807 19:49:43.358917 8856 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:43.251886 12123 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:43.363649 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.409674 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:43.410562 9020 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:49:43.387239 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:43.412508 9027 raft_consensus.cc:2393] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:43.376772 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.377230 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.421545 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.434804 9017 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:43.425503 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.423780 9022 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:43.362659 12072 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:43.285627 11630 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.350888 12118 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.415642 11797 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:43.371438 12138 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:43.443849 9021 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:43.392305 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.371359 11107 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.460963 9020 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:43.463670 11827 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:43.465804 12055 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.451941 12072 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:43.468825 11107 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.482110 11967 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:43.483528 11107 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.464360 11827 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:43.484328 11827 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.486696 11827 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:43.476451 12072 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:43.461809 11797 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.459153 12063 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.505712 9015 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:43.510993 12063 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.518416 12157 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.520833 12123 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.462400 12138 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:43.538381 12080 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.539198 12157 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.543120 11797 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:43.550155 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.557350 12138 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:43.503940 12148 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.557744 12118 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:43.559976 8776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1481) took 2398 ms (client timeout 2078 ms). Trace:
W20260807 19:49:43.594508 8776 rpcz_store.cc:269] 0807 19:49:41.161531 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.161941 (+ 410us) service_pool.cc:224] Handling call
0807 19:49:43.559686 (+2397745us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000afd54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:43.559876 (+ 190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350238976}
W20260807 19:49:43.564994 9019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37380 (request call id 168) took 2073 ms (client timeout 1957 ms). Trace:
W20260807 19:49:43.595714 9019 rpcz_store.cc:269] 0807 19:49:41.491484 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.492942 (+ 1458us) service_pool.cc:224] Handling call
0807 19:49:43.564746 (+2071804us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b5400418554. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:43.564910 (+ 164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597837184}
I20260807 19:49:43.572790 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:43.573789 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1498) took 1849 ms. Trace:
I20260807 19:49:43.597664 8780 rpcz_store.cc:276] 0807 19:49:41.724919 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.725372 (+ 453us) service_pool.cc:224] Handling call
0807 19:49:41.725724 (+ 352us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:41.868112 (+142388us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:41.868144 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:41.868189 (+ 45us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:49:41.868218 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:43.573497 (+1705279us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b5400327b54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:43.573556 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:43.573718 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326973824}
I20260807 19:49:43.566785 12027 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:43.612335 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.594444 12080 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:43.624903 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 490072us
I20260807 19:49:43.617192 12091 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:43.633265 9850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s user 0.012s sys 0.021s
W20260807 19:49:43.633759 9850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s user 0.012s sys 0.021s
W20260807 19:49:43.653091 8789 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s user 0.024s sys 0.029s
W20260807 19:49:43.653640 8789 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s user 0.024s sys 0.029s
W20260807 19:49:43.636101 11942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s user 0.008s sys 0.033s
W20260807 19:49:43.665045 8854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s user 0.036s sys 0.029s
W20260807 19:49:43.665517 8854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s user 0.036s sys 0.029s
I20260807 19:49:43.669725 8970 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:43.671104 8972 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:43.678138 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.699862 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:43.702047 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:43.660235 11942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s user 0.008s sys 0.034s
I20260807 19:49:43.703980 12063 raft_consensus.cc:2764] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260807 19:49:43.704675 12063 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:43.705214 12063 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.707480 12063 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.707989 12103 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:43.698829 12084 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:43.713946 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.715067 8903 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:43.636631 12091 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.741271 8901 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:43.657796 12056 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260807 19:49:43.720232 11797 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.719079 12118 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.718899 12175 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.724392 11926 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.700071 11957 consensus_queue.cc:1059] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260807 19:49:43.721674 12176 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:43.780117 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:43.744700 12055 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.725229 12084 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.751812 11965 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:43.823510 11761 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260807 19:49:43.811059 12148 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:43.825281 12132 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.743s user 0.024s sys 0.029s
I20260807 19:49:43.821746 11970 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:43.793857 11797 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.845501 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:43.833834 12132 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.752s user 0.024s sys 0.029s
W20260807 19:49:43.831856 11389 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s user 0.012s sys 0.019s
I20260807 19:49:43.855221 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.811477 12175 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.864279 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:43.864578 9019 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:43.823261 12055 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.822866 12176 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.811650 12118 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:43.851380 12148 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:43.883137 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:43.852144 11926 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:43.875891 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:43.832954 11965 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:43.868670 11389 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.571s user 0.012s sys 0.019s
I20260807 19:49:43.881608 9021 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:43.871819 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.890468 9018 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260807 19:49:43.891158 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 437934us
I20260807 19:49:44.008848 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1516) took 1025 ms. Trace:
I20260807 19:49:44.009814 8789 rpcz_store.cc:276] 0807 19:49:42.983371 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:43.083580 (+100209us) service_pool.cc:224] Handling call
0807 19:49:43.083983 (+ 403us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:43.111625 (+ 27642us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:43.111662 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:43.112144 (+ 482us) log.cc:844] Serialized 26 byte log entry
0807 19:49:43.657092 (+544948us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:43.657134 (+ 42us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:43.657206 (+ 72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:44.007740 (+350534us) raft_consensus.cc:1685] finished
0807 19:49:44.007834 (+ 94us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:44.008447 (+ 613us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":544576,"threads_started":1,"wal-append.queue_time_us":770062}
W20260807 19:49:44.013943 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:44.020797 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:44.024744 8856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s user 0.007s sys 0.018s
W20260807 19:49:44.025576 8856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s user 0.007s sys 0.018s
W20260807 19:49:44.026160 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 591872us
I20260807 19:49:43.891229 8853 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:44.031332 8856 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.033710 8856 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.037868 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.039297 9022 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
W20260807 19:49:43.935075 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 341155us
I20260807 19:49:44.049077 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.050150 9027 raft_consensus.cc:2368] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:44.051508 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.921782 10988 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.053534 10988 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.053921 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.055442 8904 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:43.944233 11664 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:44.059499 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 168196us
I20260807 19:49:44.061156 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.047897 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.065364 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:43.952462 11797 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:43.892328 11926 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.071024 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.072902 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.073513 11761 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:44.077180 8906 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:44.079067 11797 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.080124 11797 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.060245 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.085592 11926 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.088742 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:43.874049 11982 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.894670 12056 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:43.968972 12055 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:43.951211 12072 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.117620 12056 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.888134 12148 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:43.976395 12138 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:43.970758 12175 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:43.965058 12187 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.050s user 0.000s sys 0.001s
I20260807 19:49:44.134359 12148 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:43.872018 12163 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.135277 12138 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.135993 12163 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.138842 12163 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:44.120930 11965 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.122067 12090 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:44.111310 11797 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.109606 9023 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:44.065735 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.095194 11982 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.144090 12066 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:44.148152 9008 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:44.100124 11970 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260807 19:49:44.148844 9022 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 19:49:44.149195 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.142092 12090 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:44.151639 12120 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:44.170228 9027 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:44.171921 12090 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:44.173919 12175 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:44.155918 12066 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:44.174916 12175 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.175469 12066 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.169895 11982 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.151214 12072 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.179626 9017 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:44.160995 12138 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.180331 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.186079 11935 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:44.194339 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.195050 12087 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.189213 12175 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.196022 12087 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.198971 12087 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.187395 12056 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:44.208680 9023 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:44.213757 8903 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260807 19:49:44.217744 11967 consensus_meta.cc:287] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.735s user 0.007s sys 0.011s
I20260807 19:49:44.217538 11812 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:44.218758 8779 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:44.207104 12138 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.221524 11930 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.223686 11930 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:44.221969 8779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53512 (request call id 119) took 2520 ms (client timeout 1644 ms). Trace:
W20260807 19:49:44.225528 8779 rpcz_store.cc:269] 0807 19:49:41.701443 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.703551 (+ 2108us) service_pool.cc:224] Handling call
0807 19:49:44.221703 (+2518152us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b5400263254. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:44.221827 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5773373184}
I20260807 19:49:44.231699 12090 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.233597 11745 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.234449 11745 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.235728 12090 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.237500 11745 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.241704 12091 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.235751 12084 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.243624 11812 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:44.244643 12091 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:44.244539 11812 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.245316 12118 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.245472 12091 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.247695 12100 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:44.248834 12091 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.249508 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.250658 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.251647 12091 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:44.252821 8911 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:44.253746 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:44.255105 11107 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.255311 8906 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.255718 11107 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.251647 8904 raft_consensus.cc:2368] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6.
I20260807 19:49:44.224962 12138 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.262856 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.254731 12091 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.255081 11817 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.268572 11817 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.269038 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:44.271014 11817 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.272111 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.272677 8906 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:44.275292 11389 raft_consensus.cc:3055] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:44.275837 11389 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:44.276158 11761 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:44.277128 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.278220 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.279122 8790 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:44.280292 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:44.283815 12080 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:44.286268 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.288452 11389 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:44.290091 12222 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.145s user 0.001s sys 0.000s
I20260807 19:49:44.290969 11524 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.292905 11389 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.274838 12091 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.294773 12108 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.291841 11524 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.299269 11524 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:44.293911 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.300310 8790 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:44.300607 12108 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:49:44.301681 12224 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.135s user 0.000s sys 0.004s
I20260807 19:49:44.301447 12108 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.273826 10988 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.304162 12108 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.276494 12066 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.305334 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.307191 12066 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:44.309888 8910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 1048) took 1511 ms (client timeout 1922 ms). Trace:
W20260807 19:49:44.310367 8910 rpcz_store.cc:269] 0807 19:49:42.800117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:42.800730 (+ 613us) service_pool.cc:224] Handling call
0807 19:49:44.309405 (+1508675us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b54003d68d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:44.309638 (+ 233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070753664}
I20260807 19:49:44.315047 11930 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.318936 11630 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.321928 11545 raft_consensus.cc:3055] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:44.322474 11545 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:44.323197 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.308779 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.324362 8780 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:44.323624 11545 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.328832 11545 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.336439 12027 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.302105 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1215) took 1765 ms. Trace:
I20260807 19:49:44.339172 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.353021 12027 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.354882 12176 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.340865 9019 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:44.342295 11107 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.342976 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.356158 9022 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:44.345490 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:44.348268 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.357487 9892 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:44.355454 12027 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.351194 8789 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260807 19:49:44.359429 8789 pending_rounds.cc:85] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:44.359782 8789 pending_rounds.cc:107] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260807 19:49:44.360229 8789 raft_consensus.cc:2889] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:44.352944 8775 rpcz_store.cc:276] 0807 19:49:42.537058 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:42.571819 (+ 34761us) service_pool.cc:224] Handling call
0807 19:49:42.815217 (+243398us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:42.815605 (+ 388us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:42.815635 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:42.816121 (+ 486us) log.cc:844] Serialized 26 byte log entry
0807 19:49:42.939466 (+123345us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:42.939627 (+ 161us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:42.939776 (+ 149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:43.212084 (+272308us) raft_consensus.cc:1685] finished
0807 19:49:43.212107 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:44.301619 (+1089512us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54000974d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:44.301872 (+ 253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3623,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6912,"spinlock_wait_cycles":558993664,"thread_start_us":120201,"threads_started":1,"wal-append.queue_time_us":301027,"wal-append.run_cpu_time_us":2235,"wal-append.run_wall_time_us":1062507}
I20260807 19:49:44.364985 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.365557 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.365782 9019 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:44.349440 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.367286 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.365458 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.333825 8780 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:44.369472 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.370219 9018 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:44.371083 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.371390 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.372099 8906 raft_consensus.cc:2393] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
W20260807 19:49:44.368180 12221 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.217s user 0.000s sys 0.003s
I20260807 19:49:44.368806 8910 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:44.373181 11761 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:44.373684 8629 catalog_manager.cc:5697] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:44.364154 10988 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:44.378815 8629 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 5 to 7, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 7 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:44.380153 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.380745 8629 catalog_manager.cc:5697] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 4 to 6, leader changed from 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:44.383128 12113 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.383716 12113 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.386186 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.386816 12113 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.388089 8787 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260807 19:49:44.388576 8787 pending_rounds.cc:85] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:44.388833 8787 pending_rounds.cc:107] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:44.389067 8787 raft_consensus.cc:2889] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 19:49:44.390408 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.391450 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1515) took 1408 ms. Trace:
I20260807 19:49:44.391811 8787 rpcz_store.cc:276] 0807 19:49:42.982980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:43.022791 (+ 39811us) service_pool.cc:224] Handling call
0807 19:49:43.023195 (+ 404us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:44.389583 (+1366388us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:44.391146 (+ 1563us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540023ebd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:44.391389 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3137562496}
I20260807 19:49:44.395062 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.395931 9022 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:44.382282 8629 catalog_manager.cc:5697] T bf699a9379e64939a7048af91aef93ec P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:44.398759 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.399942 9022 raft_consensus.cc:2376] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6. Re-sending same reply.
I20260807 19:49:44.400527 8910 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:44.400944 12113 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.401746 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.401625 12113 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.402724 8901 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260807 19:49:44.403158 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.402486 8904 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:44.403424 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.404174 8908 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:44.404883 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.404287 8911 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:44.405746 8904 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:44.406905 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.405824 8902 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:44.407666 8910 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:44.407387 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.408890 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.409188 8904 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Handling vote request from an unknown peer 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:44.408455 8911 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:49:44.409811 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.403128 8901 pending_rounds.cc:85] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 2
I20260807 19:49:44.409582 8902 raft_consensus.cc:1275] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 19:49:44.411439 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.412461 8901 pending_rounds.cc:107] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260807 19:49:44.411429 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.413017 8901 raft_consensus.cc:2889] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:44.412242 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.406495 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.412319 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:44.418897 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.419416 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.420085 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.420836 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.418721 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.409659 8904 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:44.423471 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.415144 12157 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:44.424890 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.424772 8779 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:44.427935 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.428243 8787 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:44.407229 11812 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.433597 8902 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260807 19:49:44.432018 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.430965 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.436285 8780 raft_consensus.cc:2410] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:44.435686 8913 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:44.438668 9015 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.439574 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.443475 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.445206 8902 raft_consensus.cc:1275] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:44.445585 12113 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.446906 9008 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 19:49:44.448825 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.450139 8915 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:44.428722 8629 catalog_manager.cc:5697] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 2 to 4, term changed from 3 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to <none>, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New cstate: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:44.451421 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:44.454933 9022 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:44.455752 8968 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.459264 12100 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:44.459705 12100 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.460106 12100 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:44.464951 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:44.465621 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.466845 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.467846 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:44.468082 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.468649 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.469792 8775 raft_consensus.cc:2410] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:44.466719 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.472455 8783 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:44.474911 8906 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:44.476274 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.476562 12113 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:44.478412 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.480189 9018 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:44.481501 8740 leader_election.cc:400] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:44.482465 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.484967 8740 leader_election.cc:403] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260807 19:49:44.485543 8779 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:49:44.484457 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.486235 9015 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:44.488385 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.493291 8788 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:44.494735 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.494974 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.494885 8629 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:44.495904 9008 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:44.496037 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.497179 8968 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.497757 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.498447 8790 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:44.498447 9008 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.499573 12055 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:49:44.500469 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.499591 8968 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.505784 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.507475 8629 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 3 to 4, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 4 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:44.500186 8789 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:44.478528 12113 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:44.481901 12027 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:44.506551 9021 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:44.506973 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.512450 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.514482 8787 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.508184 12157 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.503345 11630 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:44.518667 11630 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.519512 12157 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.520083 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.521301 11630 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.522401 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:44.525466 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 44827.5us
I20260807 19:49:44.527040 9015 raft_consensus.cc:2410] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260807 19:49:44.526897 8856 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.533071 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:44.522293 10988 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:44.503468 12055 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.538865 8629 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:44.548427 11107 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:44.541967 11630 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:49:44.539471 12157 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260807 19:49:44.554675 8904 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:44.550942 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:44.557358 8908 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:44.545223 12055 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.561148 9021 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:44.540423 10988 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.546697 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:44.573154 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.554505 11107 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.565227 10988 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:44.576864 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:44.572064 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.596612 8904 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:44.596930 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.597605 8908 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:44.571775 11930 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:44.612927 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 79997us
W20260807 19:49:44.634469 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 71241.7us
I20260807 19:49:44.577656 12113 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:44.637238 11630 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:44.645285 11630 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.647260 12201 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260807 19:49:44.636277 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 45322.2us
I20260807 19:49:44.654976 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:44.648244 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:44.648763 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.597879 11107 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.684773 8783 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:44.684721 8788 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 19:49:44.687078 9020 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.689054 8904 raft_consensus.cc:2393] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:44.637516 9885 raft_consensus.cc:2410] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260807 19:49:44.692538 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.696972 8902 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:44.695729 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:44.702167 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:44.647809 11630 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:44.649026 11461 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election won for term 7
W20260807 19:49:44.714710 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.721197 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.745016 9022 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:44.703941 11930 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:44.720623 12027 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:44.714931 11545 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260807 19:49:44.743800 12140 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:44.721768 11461 raft_consensus.cc:697] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:44.755721 11486 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:44.761600 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:44.764276 11486 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:44.766032 12233 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260807 19:49:44.768249 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.772435 11486 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:44.780028 12229 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.146s user 0.001s sys 0.000s
I20260807 19:49:44.780526 11486 raft_consensus.cc:2764] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260807 19:49:44.738729 12113 consensus_queue.cc:1059] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260807 19:49:44.752888 12027 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:44.790167 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:44.776684 12231 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.154s user 0.001s sys 0.000s
I20260807 19:49:44.809017 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:44.810606 8783 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:44.788069 12176 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:44.815052 8790 pending_rounds.cc:85] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:44.819373 8790 pending_rounds.cc:107] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260807 19:49:44.793761 12140 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:44.817986 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:44.812186 11389 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.517s user 0.010s sys 0.002s
W20260807 19:49:44.821907 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.823479 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:44.819684 8790 raft_consensus.cc:2889] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:44.826013 8780 raft_consensus.cc:2410] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260807 19:49:44.832274 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 91179.9us
I20260807 19:49:44.795377 12163 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260807 19:49:44.790076 11630 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260807 19:49:44.841216 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.842460 12110 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:44.848122 8783 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:44.854831 8968 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:44.858222 8968 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.862391 11983 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:44.862967 11983 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:44.863332 11983 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:44.860872 12041 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:44.869091 12041 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:44.870980 8972 leader_election.cc:400] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6.
I20260807 19:49:44.871412 8972 leader_election.cc:403] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:44.825075 12100 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.877400 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:44.881346 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:44.869542 12041 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:44.871341 12110 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:44.891924 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:44.923635 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:44.890164 11915 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:44.870741 9023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:44.944947 9023 raft_consensus.cc:606] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
W20260807 19:49:44.945238 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 59348.3us
I20260807 19:49:44.946393 9023 raft_consensus.cc:614] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:44.922056 12201 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:44.899786 11486 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:44.954027 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:44.966977 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 75153.8us
I20260807 19:49:44.951799 11915 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:44.955999 11930 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:44.969502 11107 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:49:44.988114 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 137032us
I20260807 19:49:44.893920 11461 consensus_queue.cc:237] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:44.990626 9020 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:44.986697 11630 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:44.991133 11107 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:44.944797 12140 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:45.005218 9008 consensus_meta.cc:287] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.502s user 0.007s sys 0.003s
I20260807 19:49:45.006153 9008 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:44.997380 12103 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6.
I20260807 19:49:45.002983 11630 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:45.005535 11107 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:45.015022 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 241711us
I20260807 19:49:45.015277 12176 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260807 19:49:45.016196 11630 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.015556 12100 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:45.033699 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:45.023181 11915 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.039088 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.041556 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:45.044049 12234 log.cc:927] Time spent T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.255s user 0.002s sys 0.000s
I20260807 19:49:45.022691 10988 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:45.064980 8787 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.531s user 0.011s sys 0.000s
I20260807 19:49:45.065567 8787 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:49:45.073284 12241 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.258s user 0.003s sys 0.000s
W20260807 19:49:45.075975 12239 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.109s user 0.001s sys 0.001s
I20260807 19:49:45.083432 10988 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.086059 11983 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.079047 12201 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 19:49:45.105204 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 159148us
W20260807 19:49:45.106850 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 82652us
W20260807 19:49:45.111284 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 69832us
I20260807 19:49:45.112226 8856 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.101645 12201 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:45.079103 12248 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260807 19:49:45.109366 12055 consensus_meta.cc:287] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.523s user 0.013s sys 0.000s
I20260807 19:49:45.117961 12201 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.088169 11915 consensus_queue.cc:1059] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.024s
I20260807 19:49:45.120682 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.128527 12140 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260807 19:49:45.116803 11983 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.122882 12055 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.110882 11486 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:45.136224 11630 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.139036 11930 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:45.142362 11827 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:45.141294 11107 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.137912 11486 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.147658 12260 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:45.137048 12055 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.146997 11827 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.155678 11107 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.155977 11827 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.143741 11630 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:45.169927 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 60848.5us
W20260807 19:49:45.174831 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51149.6us
W20260807 19:49:45.181001 12273 log.cc:927] Time spent T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260807 19:49:45.184676 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:45.195009 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 50702.2us
I20260807 19:49:45.195945 8856 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.197052 12248 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:45.197546 12248 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:45.197921 12248 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:45.187083 11827 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:45.281297 12268 log.cc:927] Time spent T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.091s user 0.000s sys 0.001s
I20260807 19:49:45.224383 12041 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.466033 12041 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:45.287295 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:45.466774 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 251801us
W20260807 19:49:45.467259 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:45.327425 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:45.439107 12152 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:45.468384 12152 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:45.469030 12248 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.469684 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:45.470778 12152 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.471122 12248 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:45.443410 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:45.443747 8849 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.474289 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.476572 12276 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:45.477187 12276 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.478271 12276 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.480614 12260 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:45.446612 8856 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.481997 12260 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:45.483840 12260 raft_consensus.cc:697] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.484803 12260 consensus_queue.cc:237] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.173542 12271 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:45.448630 8972 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:45.490345 8972 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:45.492022 8972 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:45.450201 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 255482us
I20260807 19:49:45.495625 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.496476 12136 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:45.455942 12080 raft_consensus.cc:3055] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
W20260807 19:49:45.457428 12238 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.701s user 0.012s sys 0.000s
I20260807 19:49:45.245553 12266 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:45.463102 12055 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:45.483136 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:45.447677 12027 consensus_meta.cc:287] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.642s user 0.022s sys 0.002s
W20260807 19:49:45.449642 12275 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.291s user 0.000s sys 0.009s
I20260807 19:49:45.492780 12102 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:45.493117 11545 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:45.551913 8787 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
W20260807 19:49:45.492693 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 46167.5us
I20260807 19:49:45.496910 12152 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:45.497362 12113 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:45.561965 12113 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:45.562434 12113 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:45.497709 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:45.455019 10988 raft_consensus.cc:3055] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:45.524240 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:45.527937 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.540932 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:45.541716 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:45.567426 8788 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:45.555467 8856 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.557511 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:45.560583 12152 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:45.552071 12027 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.440196 12201 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.553162 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:45.569599 8776 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:45.526710 12080 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:45.557489 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.534870 12271 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.542937 12266 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:45.566033 10988 raft_consensus.cc:740] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:45.588483 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 60663.3us
I20260807 19:49:45.580909 8856 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.588665 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:45.597280 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 44264.1us
I20260807 19:49:45.598184 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:45.598831 8783 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:45.581892 12260 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:45.548027 12100 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.038s
I20260807 19:49:45.552654 12055 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.609442 9015 raft_consensus.cc:1240] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:45.615025 8628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38434 (request call id 180) took 9005 ms. Trace:
I20260807 19:49:45.616180 12055 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.588956 12266 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.589382 12248 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:49:45.596437 12103 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:45.626524 9022 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:45.627171 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:45.634405 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:45.636633 9021 raft_consensus.cc:2368] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
W20260807 19:49:45.637513 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 115929us
W20260807 19:49:45.656594 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 45097.7us
I20260807 19:49:45.624735 12260 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.659296 12260 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.629935 12103 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:45.671790 12103 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:45.667053 12056 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 19:49:45.671818 12080 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.677067 12080 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.675521 12293 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:45.681133 10988 consensus_queue.cc:260] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.682240 10988 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:45.683435 12103 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.684803 8780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "300f918f185249e6895ab57edd060d31"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:60644
I20260807 19:49:45.685426 8780 raft_consensus.cc:606] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Received request to transfer leadership
I20260807 19:49:45.685735 12103 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.685735 8780 raft_consensus.cc:614] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:45.671702 8783 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:45.688855 12271 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:45.692781 9838 leader_election.cc:336] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.644s (SENT)
I20260807 19:49:45.694308 12293 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.695454 12293 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:45.697257 8740 connection.cc:256] RPC call timeout handler was delayed by -1.3336860248818994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
I20260807 19:49:45.698484 8972 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:45.670122 12248 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:45.711311 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.615499 8628 rpcz_store.cc:276] 0807 19:49:36.609952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:36.612579 (+ 2627us) service_pool.cc:224] Handling call
0807 19:49:36.897371 (+284792us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b540011a8d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56e470 00007f687b60d31a 00007f68813fd38c 00007f68814f9bfe 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc 00007f687f9b2592 00007f687993e07c 00007f687993d4b6
0807 19:49:45.614788 (+8717417us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:41.550302 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:41.550875 (+ 573us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:41.550907 (+ 32us) write_op.cc:435] Acquired schema lock
0807 19:49:41.550922 (+ 15us) tablet.cc:662] Decoding operations
0807 19:49:41.551829 (+ 907us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:41.551891 (+ 62us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:41.551917 (+ 26us) tablet.cc:685] Acquiring locks for 9 operations
0807 19:49:41.598313 (+ 46396us) tablet.cc:701] Row locks acquired
0807 19:49:41.598342 (+ 29us) write_op.cc:260] PREPARE: finished
0807 19:49:41.598522 (+ 180us) write_op.cc:270] Start()
0807 19:49:41.598666 (+ 144us) write_op.cc:276] Timestamp: P: 1786132181598474 usec, L: 0
0807 19:49:41.598691 (+ 25us) op_driver.cc:348] REPLICATION: starting
0807 19:49:41.603292 (+ 4601us) log.cc:844] Serialized 3613 byte log entry
0807 19:49:42.650189 (+1046897us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:43.002300 (+352111us) op_driver.cc:464] REPLICATION: finished
0807 19:49:43.933104 (+930804us) write_op.cc:301] APPLY: starting
0807 19:49:43.935493 (+ 2389us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:43.957561 (+ 22068us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:43.958706 (+ 1145us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:43.968494 (+ 9788us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:43.968743 (+ 249us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0807 19:49:43.968794 (+ 51us) write_op.cc:312] APPLY: finished
0807 19:49:43.969101 (+ 307us) log.cc:844] Serialized 70 byte log entry
0807 19:49:44.167601 (+198500us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:44.168123 (+ 522us) write_op.cc:454] Released schema lock
0807 19:49:44.168303 (+ 180us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:44.251534 (+ 83231us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b18000ec1a0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f687b1ab6ac 00007f68784e0e3b 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687f05a8d0 00007f687f05ae69 00007f687f05d0fa
0807 19:49:44.251676 (+ 142us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:44.251706 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":130786,"reported_tablets":43,"spinlock_wait_cycles":92568320,"child_traces":[["op",{"apply.queue_time_us":916281,"apply.run_cpu_time_us":6778,"apply.run_wall_time_us":238371,"num_ops":9,"prepare.queue_time_us":61690,"prepare.run_cpu_time_us":16386,"prepare.run_wall_time_us":146433,"raft.queue_time_us":291122,"raft.run_cpu_time_us":292710,"raft.run_wall_time_us":1614285,"replication_time_us":1383388,"spinlock_wait_cycles":76377216,"thread_start_us":1207925,"threads_started":6,"wal-append.queue_time_us":378293,"wal-append.run_cpu_time_us":38483,"wal-append.run_wall_time_us":2090844}]]}
I20260807 19:49:45.702325 12084 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:45.742292 9838 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.710233 11630 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:45.744299 8788 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:45.708621 12110 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260807 19:49:45.709164 12177 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260807 19:49:45.751215 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 68) took 4317 ms. Trace:
W20260807 19:49:45.756335 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 55901.4us
I20260807 19:49:45.758469 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:45.760200 8968 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.716089 12248 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.758996 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:45.751353 12041 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:45.756295 12084 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.771544 8783 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:45.756392 12027 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.757891 12103 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:45.760403 12136 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260807 19:49:45.783509 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:45.787657 12056 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:45.786371 12041 raft_consensus.cc:697] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:45.787492 8630 catalog_manager.cc:5697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:45.768421 12296 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:45.774120 12201 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.761431 11630 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.760191 8629 rpcz_store.cc:276] 0807 19:49:41.427929 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.449748 (+ 21819us) service_pool.cc:224] Handling call
0807 19:49:45.744951 (+4295203us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:45.084409 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:45.085090 (+ 681us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:45.085143 (+ 53us) write_op.cc:435] Acquired schema lock
0807 19:49:45.085177 (+ 34us) tablet.cc:662] Decoding operations
0807 19:49:45.086142 (+ 965us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:45.086245 (+ 103us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:45.086274 (+ 29us) tablet.cc:685] Acquiring locks for 8 operations
0807 19:49:45.087035 (+ 761us) tablet.cc:701] Row locks acquired
0807 19:49:45.087071 (+ 36us) write_op.cc:260] PREPARE: finished
0807 19:49:45.087272 (+ 201us) write_op.cc:270] Start()
0807 19:49:45.087450 (+ 178us) write_op.cc:276] Timestamp: P: 1786132185087208 usec, L: 0
0807 19:49:45.087484 (+ 34us) op_driver.cc:348] REPLICATION: starting
0807 19:49:45.111259 (+ 23775us) log.cc:844] Serialized 3283 byte log entry
0807 19:49:45.195599 (+ 84340us) op_driver.cc:464] REPLICATION: finished
0807 19:49:45.582112 (+386513us) write_op.cc:301] APPLY: starting
0807 19:49:45.593853 (+ 11741us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:45.648893 (+ 55040us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:45.658616 (+ 9723us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:45.659857 (+ 1241us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:45.660188 (+ 331us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0807 19:49:45.660250 (+ 62us) write_op.cc:312] APPLY: finished
0807 19:49:45.660611 (+ 361us) log.cc:844] Serialized 64 byte log entry
0807 19:49:45.660796 (+ 185us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:45.660926 (+ 130us) write_op.cc:454] Released schema lock
0807 19:49:45.660971 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:45.661435 (+ 464us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:45.661466 (+ 31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":198916,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":358608,"apply.run_cpu_time_us":3735,"apply.run_wall_time_us":84337,"num_ops":8,"prepare.queue_time_us":160933,"prepare.run_cpu_time_us":4826,"prepare.run_wall_time_us":53086,"replication_time_us":107889,"thread_start_us":530695,"threads_started":2}]]}
I20260807 19:49:45.796294 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.785981 12103 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.816597 8630 catalog_manager.cc:5697] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:45.785655 12136 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:45.823024 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 165615us
I20260807 19:49:45.811744 12113 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:45.819847 12302 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.056s user 0.001s sys 0.000s
I20260807 19:49:45.789896 12056 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:45.839314 8783 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:49:45.810366 12152 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:45.844688 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 197387us
W20260807 19:49:45.848510 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 150107us
W20260807 19:49:45.837278 12301 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
I20260807 19:49:45.833513 12100 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:45.835049 12136 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.854449 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.852033 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.861784 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.861574 8630 catalog_manager.cc:5697] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 4, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 4 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:45.862816 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.855106 12201 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260807 19:49:45.789737 12041 consensus_queue.cc:237] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.863801 8630 catalog_manager.cc:5697] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 5, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:45.867036 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.871788 8630 catalog_manager.cc:5697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 4, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:45.873538 12201 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:45.873847 12113 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.873723 12248 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.874493 12113 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:45.874867 12113 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:45.875212 12113 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:45.876533 11827 consensus_queue.cc:1059] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260807 19:49:45.876768 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 0ba47e79caa54c409c53304a66ce0160 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:45.877692 12303 raft_consensus.cc:3055] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:45.878087 12303 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.878508 12103 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.876770 12261 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:45.879861 12201 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:45.879117 8856 leader_election.cc:400] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:49:45.881522 8856 leader_election.cc:403] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260807 19:49:45.882135 12316 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:45.883445 12314 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:45.855475 12100 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:45.890545 12100 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:45.890858 12306 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.100s user 0.000s sys 0.001s
I20260807 19:49:45.883579 11486 consensus_queue.cc:1059] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260807 19:49:45.893232 11486 raft_consensus.cc:3055] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:45.893638 11486 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:45.894410 11486 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.895678 11486 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:45.879334 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:45.888198 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.878902 12303 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.907409 12303 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:45.870051 12084 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.897837 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.883350 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.909003 8906 raft_consensus.cc:2393] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:45.889124 11630 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:45.899525 12323 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:45.899417 11545 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:45.911598 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.897989 12201 raft_consensus.cc:3055] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:45.912804 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:45.913355 8906 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.900239 8849 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.901654 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.902177 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.902123 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.902640 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:45.915799 8790 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:45.903272 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.916743 8902 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:45.904275 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.904592 12056 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.905292 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.918958 8904 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:45.906286 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.872219 12027 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:45.922049 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.922700 8902 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:45.910629 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.923749 8908 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:45.921692 12027 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:45.924938 8901 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:45.925635 12027 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:45.925920 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:45.926491 8901 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:45.928478 11545 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:45.929034 11545 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.929872 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:45.930644 8908 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 6
I20260807 19:49:45.930698 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.929944 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.931356 11545 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.931564 8911 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:45.932180 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.925768 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.932629 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.912724 12201 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:45.937995 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:45.938915 8776 raft_consensus.cc:2410] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:49:45.939467 9020 raft_consensus.cc:1240] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:45.940032 9022 raft_consensus.cc:1275] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:45.940882 9023 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:45.931318 8915 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:45.935021 8902 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:45.940223 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:45.936553 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:45.936942 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.937649 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:45.942567 9008 raft_consensus.cc:2410] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:45.935667 12323 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:45.943352 12176 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:45.943761 12176 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:45.944089 12176 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:45.945623 12248 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:45.945950 8849 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:45.947005 11827 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:45.947649 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.946044 12248 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:45.948361 12248 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:45.948998 12276 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:45.938900 12314 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:49:45.950505 12266 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.951870 8913 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:45.955314 12113 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.956548 8787 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:45.960610 12110 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:45.961280 12110 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.963472 12110 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:45.961586 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:45.960813 8968 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:45.963865 12100 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:45.937456 8630 catalog_manager.cc:5697] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 4 to 5, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:45.986745 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:45.993147 9850 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:45.963438 10988 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:45.955798 12136 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:46.011521 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 45899us
W20260807 19:49:46.005129 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 46898.8us
I20260807 19:49:46.014705 8901 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:46.024731 8906 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:46.047353 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.000705 12152 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 19:49:46.052779 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 92061.9us
I20260807 19:49:46.054625 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.017253 12103 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:46.059809 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 192838us
I20260807 19:49:45.961848 12201 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.017529 11545 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:46.061564 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.062671 8849 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:46.076164 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.045516 12314 raft_consensus.cc:2764] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260807 19:49:46.081655 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.083438 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.079214 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:46.098363 8787 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:46.056452 12152 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:46.070745 12341 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.053s user 0.000s sys 0.001s
I20260807 19:49:46.071151 12266 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.106796 9017 raft_consensus.cc:2393] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:46.016716 10988 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.052304 12110 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:46.072434 12055 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:46.120059 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.101751 8630 catalog_manager.cc:5697] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 2 to 3, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 3 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:46.131855 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 112660us
I20260807 19:49:46.113608 12152 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:46.139777 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 66046.8us
I20260807 19:49:46.141103 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.174327 9021 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:46.084079 11545 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.187173 12027 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:46.188460 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:46.127240 12055 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:46.118254 12102 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260807 19:49:46.092643 12103 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:46.191745 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 129197us
I20260807 19:49:46.110214 12293 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260807 19:49:46.226058 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.123509 12201 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:46.245427 9017 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:46.249459 12136 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:46.229645 12248 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.257668 12248 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.228562 12276 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:46.258704 12276 raft_consensus.cc:697] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:46.258879 12100 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:46.259891 12276 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:46.261109 11545 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:46.262955 12027 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.263761 8776 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:46.264793 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.222422 12084 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:46.268810 12084 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.271044 12084 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:46.276594 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:46.281078 12337 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
I20260807 19:49:46.285610 9027 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:46.286909 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:46.287228 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.287506 9027 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:46.288697 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.289863 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:46.290366 9018 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:46.290666 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.292210 9019 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:46.292873 9015 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:46.296049 9027 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:46.296459 8630 catalog_manager.cc:5697] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 6, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:46.300390 12341 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260807 19:49:46.304414 12140 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:46.304910 12140 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:46.305922 12140 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.307302 12140 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:46.308844 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.288601 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:46.297258 9017 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:46.329056 10988 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:46.338454 8908 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:46.361079 9020 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:46.361044 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 307742us
I20260807 19:49:46.311740 12176 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.225486 12055 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:46.339516 11630 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:46.371376 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 126112us
I20260807 19:49:46.245409 12113 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.371979 9023 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:46.193662 12110 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.397984 9018 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:46.380179 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:46.386698 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:46.401616 8790 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:46.389554 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:46.390177 12336 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:46.392450 12260 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:46.408174 8913 raft_consensus.cc:1240] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:49:46.394837 12103 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:46.396098 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.410408 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.410773 12260 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.397212 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.403374 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.405248 8776 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:46.412662 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:46.411089 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.396370 12176 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.406167 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.409076 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:46.417452 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.418853 9021 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:46.418807 8904 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:46.412555 12084 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:46.419708 8904 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:46.420634 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.424067 8910 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:49:46.422503 8906 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:46.426299 9027 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260807 19:49:46.379987 11630 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.428586 11630 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:46.419224 11486 consensus_meta.cc:287] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.523s user 0.009s sys 0.004s
I20260807 19:49:46.432253 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:46.431778 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.435824 9022 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:46.437283 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.437431 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:46.422066 12336 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:46.504360 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 70247.1us
W20260807 19:49:46.508716 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 74384.8us
I20260807 19:49:46.423058 12027 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:46.399494 12113 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260807 19:49:46.438316 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:46.511688 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 73446.8us
I20260807 19:49:46.506546 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.504941 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:46.513047 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.436790 8904 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.438207 11630 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:46.521744 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.504024 12080 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:46.503929 11486 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 19:49:46.548310 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:46.523941 8913 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:49:46.583652 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 57980.6us
W20260807 19:49:46.673573 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:46.709426 8904 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:46.712689 8908 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:46.514662 12027 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:46.524492 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.711037 8915 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:46.720204 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.721843 8790 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:46.721120 8906 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:46.517146 12336 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:46.727151 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:46.730283 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:46.590098 12348 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260807 19:49:46.713912 12080 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:46.523720 11630 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:46.736136 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.545142 12113 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:46.735913 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.597360 12110 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:46.568636 12260 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.553334 12266 raft_consensus.cc:3055] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:46.710071 11486 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:46.738741 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:46.622517 12084 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:46.744488 8915 raft_consensus.cc:2410] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260807 19:49:46.745157 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 232194us
I20260807 19:49:46.752888 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:46.737447 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 405095us
I20260807 19:49:46.515745 12176 consensus_queue.cc:1059] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.221s
I20260807 19:49:46.721455 12027 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:46.724658 11970 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:46.510893 12346 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.228s
W20260807 19:49:46.747922 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:46.589242 11827 consensus_queue.cc:1059] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.156s
I20260807 19:49:46.732720 12336 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:46.750330 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:46.751006 12266 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:46.754629 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.525086 12152 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:46.756430 8913 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:46.737941 12080 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:46.757189 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:46.745530 11630 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:46.554455 12356 consensus_queue.cc:1048] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.221s
I20260807 19:49:46.743816 12113 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:46.733207 12201 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260807 19:49:46.751593 11486 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:46.760485 8902 raft_consensus.cc:872] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }.
Pending config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }:
New config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:46.734549 12350 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260807 19:49:46.761662 12136 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:46.761801 11970 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:46.763039 12336 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:46.774791 8902 raft_consensus.cc:1579] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Could not prepare op 'term: 3 index: 4' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }.
Pending config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }:
New config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:46.781659 9015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "300f918f185249e6895ab57edd060d31"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:46.782258 9015 raft_consensus.cc:606] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:46.782588 9015 raft_consensus.cc:614] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:46.785944 12201 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:46.786403 12201 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:46.787281 12201 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.772809 12113 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:46.762633 12084 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.779006 12136 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:46.802132 11970 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.805969 12248 raft_consensus.cc:3055] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:46.806465 12248 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:46.807368 12248 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.809105 9885 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:46.812572 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:46.814046 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:46.814729 9015 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:46.822395 12084 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:46.781332 12365 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:46.823031 12201 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:46.823079 12110 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:46.804111 12152 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:46.835130 12357 log.cc:927] Time spent T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.065s user 0.000s sys 0.001s
W20260807 19:49:46.842996 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 88444us
I20260807 19:49:46.831310 12248 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:46.814347 12055 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.844237 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:46.834041 12084 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:46.791028 12266 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.894896 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.839746 12110 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:46.895955 8902 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } } }
W20260807 19:49:46.903828 12370 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260807 19:49:46.903640 12110 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.853130 12345 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:46.849408 12041 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:46.892395 12084 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:46.830200 12136 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:46.917514 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 200682us
I20260807 19:49:46.920811 8776 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:46.922111 8972 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:46.912917 12084 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:46.932219 9018 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:46.932519 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.933617 8908 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
W20260807 19:49:46.925175 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 46501.5us
I20260807 19:49:46.942675 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:46.913971 12041 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.918123 12266 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:46.945215 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.949033 8913 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:46.955179 9023 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:46.911705 12345 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:46.959991 9022 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:49:46.962365 8904 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:49:46.955950 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:46.963037 8901 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:46.966207 12383 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
W20260807 19:49:46.976887 12366 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.133s user 0.000s sys 0.001s
I20260807 19:49:46.956806 12365 raft_consensus.cc:2764] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260807 19:49:46.943311 12176 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260807 19:49:46.966308 12382 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:46.982899 8780 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:46.983670 12027 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:46.994553 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 50981.7us
I20260807 19:49:46.995939 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:46.955379 12376 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:47.001219 12382 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:49:47.012476 8787 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.001319 11630 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:47.015496 8972 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.017235 11461 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:47.017884 11461 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.022796 12260 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:49:47.023818 12303 raft_consensus.cc:3055] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:47.024188 11461 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.024214 12303 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:47.030341 11486 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.032567 8776 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:47.033802 10988 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:47.033540 12303 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:47.034797 12303 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.036290 10988 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:47.037318 12140 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:47.037696 12140 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:47.038470 12140 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.039454 12140 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.039912 8972 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.041961 12313 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:47.042472 12313 raft_consensus.cc:697] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:47.043092 11486 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.043357 12313 consensus_queue.cc:237] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.046288 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:47.048028 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.048725 8911 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:47.047605 12055 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.054670 12383 raft_consensus.cc:697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:47.055022 12376 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:47.055684 12376 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:47.055852 12383 consensus_queue.cc:237] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.058234 12376 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:47.046367 12041 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.075796 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:47.070741 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 124268us
I20260807 19:49:47.065266 12055 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:47.078747 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.080906 10988 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260807 19:49:47.083078 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.089963 10988 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:47.091764 8972 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.093043 8972 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.069727 12355 raft_consensus.cc:3055] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:47.101945 10988 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260807 19:49:47.104089 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 161516us
I20260807 19:49:47.104416 8972 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.104928 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:47.105372 10988 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.106773 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.089540 12103 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:47.107421 8776 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:49:47.109560 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 300f918f185249e6895ab57edd060d31 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
W20260807 19:49:47.110236 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.094374 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:47.093946 12084 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:47.113726 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.067946 12336 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.114574 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:47.115389 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.101188 11545 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:47.096658 12344 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:47.086081 11461 consensus_queue.cc:1059] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260807 19:49:47.116307 12344 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.116765 12344 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.117146 11461 raft_consensus.cc:3055] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:47.117540 11461 raft_consensus.cc:740] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:47.118355 11461 consensus_queue.cc:260] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.119650 11461 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:47.103645 12080 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.121356 8790 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:47.115320 8780 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:47.121429 12113 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.124249 8783 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:47.107792 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:47.094213 12102 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:47.136334 10988 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:47.138015 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.097581 12355 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:47.132687 12136 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.143003 12380 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:47.143750 12380 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.146193 12380 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.139310 12027 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.142937 12355 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.148005 12392 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:47.139493 8780 raft_consensus.cc:3060] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:47.148669 12392 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.149266 9022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:47.121315 12365 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:47.141745 12336 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.129737 12103 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.150449 12365 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:47.150846 12103 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.151455 12365 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:47.152654 12371 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.357s user 0.003s sys 0.005s
I20260807 19:49:47.147758 12102 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:47.159750 8780 raft_consensus.cc:2468] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:47.160588 12392 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.091318 12055 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.159983 12355 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.163269 12055 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.157018 8913 raft_consensus.cc:2410] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260807 19:49:47.114334 12084 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.159927 12102 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.149883 9022 raft_consensus.cc:606] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:47.167423 12344 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:47.168614 12102 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.180563 8790 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:47.181218 12345 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.187355 12080 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.188560 9022 raft_consensus.cc:1275] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260807 19:49:47.190982 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.191004 9022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:47.192430 9022 raft_consensus.cc:606] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:47.192842 9022 raft_consensus.cc:614] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 19:49:47.192544 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 86583.7us
I20260807 19:49:47.194309 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.195562 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.196506 8776 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 19:49:47.196935 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.197707 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.196539 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.198383 8788 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
I20260807 19:49:47.198616 8775 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.198855 12084 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:47.199362 12084 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:47.200302 12084 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.202942 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.203714 8789 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 7: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.204633 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.205530 8779 raft_consensus.cc:2410] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260807 19:49:47.206116 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.188109 12103 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:47.212610 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet be89699a1aff43f1a97b936b3154c2d5 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:47.236507 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.382902 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:47.374351 8968 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.269853 9022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:47.387178 9022 raft_consensus.cc:606] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:47.282675 8775 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:49:47.223794 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:47.306537 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.385227 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.389518 8779 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.392297 12313 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:47.215440 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.394189 12313 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.393824 8776 raft_consensus.cc:2368] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:49:47.394629 12313 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.235843 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:47.334316 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.395313 8788 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:47.387753 9018 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:47.217375 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.395205 8789 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:47.398062 12345 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:47.398501 12345 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.398875 12345 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:47.399654 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:47.400367 8913 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:49:47.404693 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 187413us
I20260807 19:49:47.405979 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.392331 8968 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.407959 8913 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:47.409119 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.388235 9022 raft_consensus.cc:614] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:47.411736 10988 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:47.412761 10988 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:47.378204 8849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.225s (SENT) [suppressed 20 similar messages]
I20260807 19:49:47.414094 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.415369 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.416687 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.219563 11630 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:47.234146 12113 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.413232 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.412294 8790 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.413344 10988 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.413967 8776 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260807 19:49:47.418488 8776 pending_rounds.cc:85] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:47.418754 8776 pending_rounds.cc:107] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260807 19:49:47.418987 8776 raft_consensus.cc:2889] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:47.419332 8776 pending_rounds.cc:107] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:47.419617 8776 raft_consensus.cc:2889] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:47.419831 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.420616 8903 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:47.419831 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.421080 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.421136 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.422458 8906 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:47.422392 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.422694 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:47.423166 8904 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:47.423477 8913 raft_consensus.cc:2393] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:47.423411 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.422158 8775 raft_consensus.cc:2368] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:49:47.410403 12100 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:47.424755 12100 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:47.424873 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.425197 12100 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:47.425531 8901 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:47.426070 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.427176 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.424901 8910 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:47.428184 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.428694 8778 raft_consensus.cc:1240] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:47.429263 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.430742 8776 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:47.431691 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:47.431766 12303 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:47.432472 12303 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.434269 8775 raft_consensus.cc:1217] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[3.2-3.2] Dedup: 3.2->[]
I20260807 19:49:47.434755 8775 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 19:49:47.435061 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:47.432529 8913 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:47.436255 8968 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.435654 8910 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:47.439085 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 123356us
I20260807 19:49:47.422359 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.441522 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.442884 8789 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:47.439843 9838 leader_election.cc:336] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.866s (SENT)
I20260807 19:49:47.445182 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:47.445276 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:47.214293 12103 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.233750 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:47.488561 8902 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:47.459745 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:47.489939 8915 raft_consensus.cc:2410] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:49:47.461467 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.491220 8788 raft_consensus.cc:2410] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260807 19:49:47.403932 12313 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.493244 12388 raft_consensus.cc:2804] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:47.477159 8972 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.457593 10988 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.510111 8775 raft_consensus.cc:1240] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:47.511646 10988 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.447110 12110 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260807 19:49:47.410723 12041 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
I20260807 19:49:47.434756 12303 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:47.644928 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 205101us
W20260807 19:49:47.713574 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 236487us
W20260807 19:49:47.597414 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 69865.3us
I20260807 19:49:47.715243 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:47.522945 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:47.716614 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.515304 8783 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260807 19:49:47.717511 8783 pending_rounds.cc:85] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:47.717764 8783 pending_rounds.cc:107] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:47.718017 8783 raft_consensus.cc:2889] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:47.513520 9023 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
W20260807 19:49:47.572154 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:47.723114 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:47.569752 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.720782 9022 raft_consensus.cc:2393] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:47.725575 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.721174 8972 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.567389 8910 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
W20260807 19:49:47.654742 12405 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.239s user 0.001s sys 0.000s
I20260807 19:49:47.733155 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.733899 8789 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:47.514756 12388 raft_consensus.cc:697] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:47.736073 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.736290 12303 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:47.737021 12303 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.737931 8790 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 19:49:47.738715 12350 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:47.739473 11461 consensus_queue.cc:1048] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:47.739235 12303 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.740691 8778 raft_consensus.cc:2368] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:49:47.736410 12355 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:47.740515 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.741320 12355 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.743021 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.743543 12355 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.743748 8779 raft_consensus.cc:2393] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.515196 12346 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:47.736964 12102 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 19:49:47.739346 8776 raft_consensus.cc:1275] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:47.746155 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.748250 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.749166 8780 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.751127 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.752409 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.744760 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.753460 8778 raft_consensus.cc:2376] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6. Re-sending same reply.
I20260807 19:49:47.755013 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.756855 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.757596 8776 raft_consensus.cc:2368] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:49:47.760056 8778 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:47.734760 12100 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.761224 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.761719 12380 raft_consensus.cc:3055] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:47.762281 12100 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.762738 8789 raft_consensus.cc:1275] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:47.762193 12380 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:47.761909 8776 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:47.612146 12327 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:47.513563 12344 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:47.516357 12398 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.126s user 0.000s sys 0.001s
I20260807 19:49:47.762909 12313 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.444695 12345 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.768824 12345 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.769007 12384 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:47.769552 12384 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.769936 12384 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:47.763846 8788 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260807 19:49:47.765225 8787 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:47.528612 9892 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
W20260807 19:49:47.776558 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:47.767464 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 43772.6us
I20260807 19:49:47.777012 8629 catalog_manager.cc:5697] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: leader changed from <none> to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:47.780022 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.774986 8787 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:47.782568 12110 consensus_queue.cc:1059] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:47.783731 12102 raft_consensus.cc:3055] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:47.784566 12102 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:47.786978 8789 raft_consensus.cc:2393] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.782004 8972 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:47.790868 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:47.792673 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:47.793365 9023 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.778589 8775 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:49:47.789247 8629 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 3 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 4 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:47.798226 10988 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:47.798691 10988 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.800309 10988 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:47.787039 8787 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.535354 12140 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:47.800806 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.806604 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.814062 8787 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:47.813823 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.816555 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.832461 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.744822 12346 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.837487 11915 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260807 19:49:47.846674 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 58886.9us
I20260807 19:49:47.796999 12388 consensus_queue.cc:237] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.848459 8968 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.763952 12380 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.809984 12103 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260807 19:49:47.832394 12382 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:47.858618 10988 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.821524 12385 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260807 19:49:47.879614 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 138850us
I20260807 19:49:47.880061 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.881551 8788 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:47.884734 8787 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260807 19:49:47.885331 8787 pending_rounds.cc:85] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 2
I20260807 19:49:47.885561 8787 pending_rounds.cc:107] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260807 19:49:47.885790 8787 raft_consensus.cc:2889] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:47.886102 8787 pending_rounds.cc:107] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:47.886323 8787 raft_consensus.cc:2889] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:47.895221 8856 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.904140 10988 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.906586 12313 consensus_queue.cc:1059] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:47.907568 12313 raft_consensus.cc:3055] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:47.907946 12313 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:47.908891 12313 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.910313 12313 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:47.893239 12346 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:47.891788 12136 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:47.913033 12136 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.913395 12136 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.916535 12416 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:47.877398 12113 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:47.912026 12380 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:47.922569 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.917900 12113 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.917115 12416 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.924484 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:47.926851 8790 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:47.924113 8775 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.924253 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.924844 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.929607 8780 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:47.927217 8788 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:49:47.941109 8849 leader_election.cc:336] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.567s (SENT)
I20260807 19:49:47.941231 12113 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.932847 12355 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:47.942062 12355 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:47.942498 12355 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.840566 12102 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.934963 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.950197 8787 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:47.926673 11461 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:47.937219 12414 log.cc:927] Time spent T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.103s user 0.000s sys 0.001s
I20260807 19:49:47.938618 12384 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.939285 12416 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.943246 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:47.959729 8788 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 8
I20260807 19:49:47.958164 11461 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:47.968439 11461 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:47.956701 8775 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:47.958137 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38436 (request call id 196) took 6433 ms. Trace:
I20260807 19:49:47.947995 9023 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:47.975144 8787 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
W20260807 19:49:47.977728 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 149039us
I20260807 19:49:47.979840 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:47.982007 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:47.985734 12410 log.cc:927] Time spent T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.095s user 0.003s sys 0.000s
I20260807 19:49:47.988617 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:47.991523 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:47.998927 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:48.000499 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 128223us
I20260807 19:49:47.997361 8788 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:47.972914 8630 rpcz_store.cc:276] 0807 19:49:41.519582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:41.520154 (+ 572us) service_pool.cc:224] Handling call
0807 19:49:42.562763 (+1042609us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5800001420. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c6a62 00007f6881492df9 00007f688142bd56 00007f68813fd248 00007f68814f9bfe 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc
0807 19:49:43.541807 (+979044us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b5800001420. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6ff0 00007f68813f1d36 00007f68814fa4be 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc 00007f687f9b2592 00007f687993e07c 00007f687993d4b6
0807 19:49:47.952493 (+4410686us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:46.773118 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:46.786787 (+ 13669us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:46.786844 (+ 57us) write_op.cc:435] Acquired schema lock
0807 19:49:46.786883 (+ 39us) tablet.cc:662] Decoding operations
0807 19:49:46.788018 (+ 1135us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:46.788084 (+ 66us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:46.788110 (+ 26us) tablet.cc:685] Acquiring locks for 8 operations
0807 19:49:46.912387 (+124277us) tablet.cc:701] Row locks acquired
0807 19:49:46.912413 (+ 26us) write_op.cc:260] PREPARE: finished
0807 19:49:46.912581 (+ 168us) write_op.cc:270] Start()
0807 19:49:46.914967 (+ 2386us) write_op.cc:276] Timestamp: P: 1786132186912534 usec, L: 0
0807 19:49:46.916587 (+ 1620us) op_driver.cc:348] REPLICATION: starting
0807 19:49:46.993027 (+ 76440us) log.cc:844] Serialized 3233 byte log entry
0807 19:49:47.391628 (+398601us) op_driver.cc:464] REPLICATION: finished
0807 19:49:47.414327 (+ 22699us) write_op.cc:301] APPLY: starting
0807 19:49:47.414421 (+ 94us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:47.416068 (+ 1647us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:47.416191 (+ 123us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:47.597668 (+181477us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:47.700486 (+102818us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0807 19:49:47.708418 (+ 7932us) write_op.cc:312] APPLY: finished
0807 19:49:47.708791 (+ 373us) log.cc:844] Serialized 64 byte log entry
0807 19:49:47.709023 (+ 232us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:47.709175 (+ 152us) write_op.cc:454] Released schema lock
0807 19:49:47.709234 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:47.709776 (+ 542us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:47.709805 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19163,"reported_tablets":43,"spinlock_wait_cycles":45967872,"child_traces":[["op",{"apply.queue_time_us":22421,"apply.run_cpu_time_us":4232,"apply.run_wall_time_us":295397,"num_ops":8,"prepare.queue_time_us":386513,"prepare.run_cpu_time_us":32982,"prepare.run_wall_time_us":339899,"raft.queue_time_us":228327,"raft.run_cpu_time_us":3725,"raft.run_wall_time_us":52178,"replication_time_us":474862,"thread_start_us":391183,"threads_started":4,"wal-append.queue_time_us":165520}]]}
I20260807 19:49:48.019809 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.028158 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:47.987049 12102 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:48.032073 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:48.028810 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:47.989671 11461 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:48.035743 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 140278us
I20260807 19:49:47.989836 12055 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:48.041044 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.045106 8968 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.000016 12303 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:48.048031 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:48.047932 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 97914.7us
I20260807 19:49:48.058420 11486 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:49:48.059083 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet fe08dfdbfbdb4fd08342efd539590bf2 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:48.059695 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260807 19:49:48.061936 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:49:48.063032 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:48.066463 12416 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:48.067445 12416 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:48.049786 11630 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:48.072443 12055 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.075163 12055 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.027671 12350 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:48.034902 12113 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:49:48.101631 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56593us
I20260807 19:49:48.102622 10988 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.103173 8968 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:48.105273 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 57299.2us
I20260807 19:49:48.106388 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.103240 10988 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:48.027227 12345 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
I20260807 19:49:48.108038 10988 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.046759 12344 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:48.116976 11486 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:48.118718 8740 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.119644 11486 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:48.120599 12041 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.121241 12041 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:48.121914 11486 consensus_queue.cc:1059] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:48.122182 12041 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.126406 12136 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.120651 8968 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.096639 12100 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.114367 11630 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:48.145888 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.135600 12344 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:48.147497 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:48.151201 8776 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:48.136427 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 135041us
W20260807 19:49:48.158489 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 87859.9us
I20260807 19:49:48.162791 8849 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:48.160871 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 107364us
I20260807 19:49:48.146739 12385 raft_consensus.cc:2804] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:48.150359 12100 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:48.199867 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.202207 9023 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:49:48.204030 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.201570 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.205394 9020 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:48.164165 12136 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.171845 12429 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:48.215438 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.221438 8776 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:48.216239 9021 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:49:48.223868 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 100709us
I20260807 19:49:48.223759 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.225387 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:48.224938 9017 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:48.230530 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.231248 8776 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:48.231870 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.232800 8778 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:48.234413 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:48.235354 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.235584 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:48.237064 8776 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:48.209379 12385 raft_consensus.cc:697] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:48.203655 12384 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.236109 8780 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:48.239488 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.242406 8789 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:48.243266 9838 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:48.253880 11970 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260807 19:49:48.258665 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.258558 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.261287 8787 raft_consensus.cc:2393] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:48.271643 8968 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.271885 12100 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.280918 12385 consensus_queue.cc:237] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:48.281926 12423 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.145s user 0.000s sys 0.001s
W20260807 19:49:48.284046 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.261458 12419 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:48.280853 12427 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.109s user 0.000s sys 0.001s
W20260807 19:49:48.300079 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 95404.4us
I20260807 19:49:48.310438 11461 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:48.296222 12440 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.321110 12440 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:48.322135 12440 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.322610 12084 raft_consensus.cc:3055] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:48.323086 12084 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:48.278719 12344 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:48.325664 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.326107 12303 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:48.326611 12303 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.318507 12419 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.328913 8910 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:48.330175 12419 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.331449 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.332706 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.306697 8849 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.334602 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.335866 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.339493 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.339690 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.340729 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.341454 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.341596 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.332268 8911 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
W20260807 19:49:48.343364 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.344139 9017 raft_consensus.cc:2393] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:48.319851 12355 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.250279 12429 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260807 19:49:48.347949 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.349838 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.349277 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.357314 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.344944 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.340647 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.359545 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.360760 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.361505 9018 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:48.348117 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.323948 12084 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.370689 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.371441 8776 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:48.364406 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.367669 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.376027 9018 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:48.377099 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.384797 9022 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260807 19:49:48.386332 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.387960 8910 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:48.387192 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.390100 9023 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:48.387284 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.391405 8906 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:48.393932 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.394675 9021 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:48.396839 9008 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:48.398433 9008 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:48.393839 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.420081 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.420897 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.444739 12303 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:48.369299 11461 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:48.508973 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 112884us
I20260807 19:49:48.487962 8780 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:48.368120 12416 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:48.489806 9018 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
W20260807 19:49:48.489926 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 48273.6us
I20260807 19:49:48.404423 8911 raft_consensus.cc:2368] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:49:48.491127 9015 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:48.491853 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.492862 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.494019 9020 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:48.493940 8902 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
W20260807 19:49:48.497772 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 150924us
I20260807 19:49:48.487159 8915 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:48.507656 9023 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:48.487426 12429 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:48.509869 8906 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:48.510964 8968 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:48.513363 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.513698 9021 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:48.519039 9018 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
W20260807 19:49:48.525130 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 185453us
I20260807 19:49:48.526445 8849 leader_election.cc:400] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:49:48.526764 8849 leader_election.cc:403] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 19:49:48.528796 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.534196 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.555538 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.556082 9019 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:48.556378 8903 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:48.522869 12303 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:48.534873 12438 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:48.531272 11630 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:48.559087 12113 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.006s
I20260807 19:49:48.541939 12100 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:48.555063 11486 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:48.516324 12429 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.522356 12416 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.580724 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.567369 12303 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:48.579718 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.579412 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.614092 8915 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:48.578768 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.581056 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.610113 8910 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:48.579125 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.516084 11461 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: NON_VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: true } }
I20260807 19:49:48.611860 8789 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:49:48.619972 8906 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:48.604126 9027 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:48.620316 8915 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:48.608891 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:48.576790 8780 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:48.593218 8904 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:48.613622 8902 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:49:48.621008 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:48.623090 8906 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:48.621595 8901 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:48.627911 8908 raft_consensus.cc:2393] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:48.631593 12041 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:48.632171 12041 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.634362 12041 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.634485 8902 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:48.636260 8775 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260807 19:49:48.637863 12428 log.cc:927] Time spent T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.415s user 0.004s sys 0.000s
I20260807 19:49:48.593357 12100 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:48.641268 12100 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:48.654274 8910 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:48.554872 12437 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260807 19:49:48.585268 12418 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:48.662191 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 120129us
I20260807 19:49:48.583946 12459 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:48.664103 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.668608 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.670825 9018 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:48.669176 8906 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:48.675288 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:48.670900 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.615443 12303 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:48.590721 12461 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:48.612243 11486 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:48.683506 8901 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:48.679881 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.682893 10988 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:48.662375 12355 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:48.677392 11630 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:48.657968 12418 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.706071 12418 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.691102 12346 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260807 19:49:48.693427 12426 consensus_queue.cc:1059] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:48.708508 12355 raft_consensus.cc:3055] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:48.708930 12355 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:48.709842 12355 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.711108 12355 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:48.695066 11461 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:48.695531 12100 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.696051 12469 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:48.714967 12469 raft_consensus.cc:697] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:48.716281 12100 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.696942 11486 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.697742 8856 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.718146 8856 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.719135 8856 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.685321 12461 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.699263 12084 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:48.699962 12429 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:48.721354 12429 raft_consensus.cc:2764] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260807 19:49:48.701210 12416 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:48.720458 12451 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:48.722673 12461 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:48.722981 12451 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:48.723440 12451 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:48.720567 12426 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:48.724123 12426 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:48.724534 12426 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:48.680852 12437 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:48.727306 8856 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.728569 12433 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:48.729017 12433 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:48.729354 12433 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:48.715921 12469 consensus_queue.cc:237] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.723531 8915 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:48.750423 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.751238 8783 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:48.689011 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:48.755626 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 66691us
I20260807 19:49:48.760957 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.759116 8783 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260807 19:49:48.807389 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 281080us
I20260807 19:49:48.808283 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.809090 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:48.809818 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 47893us
I20260807 19:49:48.810817 12140 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:48.811663 12350 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260807 19:49:48.814652 12391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s user 0.008s sys 0.010s
I20260807 19:49:48.814994 12437 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.815145 12140 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.819211 12140 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:48.816457 12437 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:48.816021 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 45011.1us
I20260807 19:49:48.820540 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:48.820926 12391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s user 0.008s sys 0.011s
I20260807 19:49:48.820645 12437 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:48.824999 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.828830 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.829869 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.830927 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.683907 12102 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260807 19:49:48.759114 12113 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.756958 12346 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:48.844259 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.845672 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.847004 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.849604 8911 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260807 19:49:48.850342 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:48.850970 9019 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:48.842115 8968 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.857049 8856 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:48.840044 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:48.902030 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:48.934764 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:48.935158 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:48.934199 11630 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:48.862210 12447 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.081s
I20260807 19:49:48.862102 12385 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 19:49:48.936337 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 79350.8us
W20260807 19:49:48.948546 12468 log.cc:927] Time spent T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.115s user 0.000s sys 0.001s
I20260807 19:49:48.949098 12416 consensus_queue.cc:1059] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:48.950045 12416 consensus_queue.cc:1243] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer a38f4d6088cf4c8cb510200ca0555801 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260807 19:49:48.951054 12416 raft_consensus.cc:3055] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:48.951514 12416 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:48.937994 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 98046us
I20260807 19:49:48.952874 11486 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:48.953889 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.833814 12438 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.100s
I20260807 19:49:48.957640 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.966929 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:48.967406 11486 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:48.969225 11486 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.860879 12477 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:48.976176 8849 leader_election.cc:400] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:49:48.945889 12385 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:48.976855 8849 leader_election.cc:403] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:48.936097 12466 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:48.938681 12461 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:49:48.975337 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 74640.6us
I20260807 19:49:48.850438 12346 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.006568 12416 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.939725 11630 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.970893 11461 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260807 19:49:49.018810 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 161839us
I20260807 19:49:48.970710 12350 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:49.023833 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:48.944208 12426 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:48.977648 12441 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:49.026693 8788 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:48.977738 12459 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:49:49.024652 8790 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:49.030169 8788 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:49.036744 12416 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:49.037750 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 207944us
I20260807 19:49:48.998229 12461 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:49.000572 12385 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:48.996021 12313 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.035s
I20260807 19:49:49.035751 8788 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.040776 9019 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:49.043047 8788 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:49.000830 12437 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:49.025007 12442 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
W20260807 19:49:49.081024 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:49.081734 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:49.088409 8779 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260807 19:49:49.096654 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:49.089375 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 57346.7us
I20260807 19:49:49.089593 12350 consensus_queue.cc:1048] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:49.097920 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.047787 9027 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260807 19:49:49.088785 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 44568us
I20260807 19:49:49.098770 12459 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:49.100942 12459 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.101454 12459 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.103432 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.104673 12459 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.100075 8849 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.105154 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.106307 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.108101 12350 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:49.108875 12350 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:49.113688 12459 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.113006 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.118505 12459 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.120539 12466 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260807 19:49:49.044018 12469 consensus_queue.cc:1175] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a38f4d6088cf4c8cb510200ca0555801" responder_term: 7 status { last_received { term: 7 index: 4 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 19:49:49.106313 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.042140 12461 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:49.122692 8968 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:49.130633 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.130903 8788 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260807 19:49:49.131405 8788 pending_rounds.cc:85] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:49.131634 8788 pending_rounds.cc:107] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260807 19:49:49.131856 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.132002 8788 raft_consensus.cc:2889] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:49.132205 8849 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.102197 12493 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.104271 12479 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:49:49.121445 12485 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260807 19:49:49.078158 12113 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:49.145426 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:49.091666 12437 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:49.146899 11486 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:49.152959 10988 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:49:49.133836 12438 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:49.141952 11461 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:49.146492 12419 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:49.186086 12477 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.225024 8789 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:49.141027 12433 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.143422 12421 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260807 19:49:49.141317 12466 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260807 19:49:49.179538 12491 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:49.185891 11461 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:49.224026 12419 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:49.144490 12493 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:49.267743 12303 consensus_meta.cc:287] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.582s user 0.009s sys 0.002s
I20260807 19:49:49.225440 12492 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:49.252907 11461 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:49:49.315480 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 182628us
I20260807 19:49:49.317308 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.241159 12440 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260807 19:49:49.269733 12419 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:49.229354 12477 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.326186 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.338363 11461 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.326629 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.308774 12429 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.340808 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.328091 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.340891 12429 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:49.341686 8908 raft_consensus.cc:3060] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:49.343447 12501 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.343992 12501 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.347982 12477 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.349257 12459 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:49.340945 12486 log.cc:927] Time spent T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.151s user 0.000s sys 0.001s
W20260807 19:49:49.353012 12497 log.cc:927] Time spent T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
I20260807 19:49:49.354568 12501 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.356593 8783 raft_consensus.cc:1275] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:49.319742 12303 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.351303 12426 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.364995 12429 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:49.365561 12303 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.367228 12493 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.368252 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:49.321979 12346 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:49.274744 12487 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.134s user 0.001s sys 0.000s
I20260807 19:49:49.325762 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.367858 12419 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.325824 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.385732 11461 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.329530 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.388012 8901 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
W20260807 19:49:49.337302 12474 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.377s user 0.006s sys 0.000s
I20260807 19:49:49.390045 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.399852 8902 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:49.400840 8908 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:49.402093 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.406080 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.406881 8901 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:49.373482 12492 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:49.411075 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:49.419011 9885 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
W20260807 19:49:49.429207 12355 consensus_meta.cc:287] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.718s user 0.008s sys 0.004s
I20260807 19:49:49.389845 12461 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.431900 12461 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.432997 8906 raft_consensus.cc:2410] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:49:49.433525 12493 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:49.436699 12419 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:49.411397 12435 consensus_queue.cc:1048] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260807 19:49:49.400945 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.444057 8902 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:49.446727 8903 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:49.449872 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:49.450537 8779 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:49.490057 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.517823 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:49.518507 8902 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:49.330993 12437 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.522801 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:49.524120 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:49.313112 12113 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.527304 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:49.412079 8911 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
W20260807 19:49:49.488171 12451 consensus_meta.cc:287] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.764s user 0.010s sys 0.001s
I20260807 19:49:49.510797 8910 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:49.518950 8853 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:49.427824 12385 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.536552 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.555734 8908 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:49.460145 12433 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.460552 12492 raft_consensus.cc:697] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:49.574396 8902 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:49.580875 8901 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
W20260807 19:49:49.582798 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 257046us
I20260807 19:49:49.529081 12380 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:49.543808 12102 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:49.572530 12461 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:49.599562 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.544693 12388 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:49.607123 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.619656 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:49.555617 12451 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.601563 12461 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:49.625073 11486 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
I20260807 19:49:49.638118 8740 leader_election.cc:400] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:49:49.641141 8740 leader_election.cc:403] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260807 19:49:49.641860 11486 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:49.643288 11486 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:49:49.644320 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.613981 12514 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:49.686547 12350 consensus_meta.cc:287] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.577s user 0.009s sys 0.002s
I20260807 19:49:49.687060 12350 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:49.687739 12492 consensus_queue.cc:237] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.690790 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1660) took 1391 ms. Trace:
I20260807 19:49:49.628911 12084 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:49.648370 12461 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:49.691284 8787 rpcz_store.cc:276] 0807 19:49:48.298980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:48.299772 (+ 792us) service_pool.cc:224] Handling call
0807 19:49:48.350801 (+ 51029us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:49:48.368279 (+ 17478us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:48.368313 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:49:48.442235 (+ 73922us) log.cc:844] Serialized 52 byte log entry
0807 19:49:48.611979 (+169744us) raft_consensus.cc:1645] Marking committed up to 2
0807 19:49:48.612026 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:48.612098 (+ 72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:49.133423 (+521325us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:49.688304 (+554881us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b240014ad28. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878384594 00007f687838269b 00007f687853e6d9 00007f687853e365 00007f687853d8c0 00007f687b1cb638 00007f687b1cbd98 00007f687b1a3588 00007f687b19e95a 00007f687b19dda6
0807 19:49:49.690497 (+ 2193us) raft_consensus.cc:1685] finished
0807 19:49:49.690524 (+ 27us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:49.690717 (+ 193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":163,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":16977,"spinlock_wait_cycles":94270592,"thread_start_us":77590,"threads_started":1,"wal-append.queue_time_us":313678}
I20260807 19:49:49.700721 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.705734 8908 raft_consensus.cc:2410] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260807 19:49:49.600580 12479 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
W20260807 19:49:49.702455 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 64477.7us
I20260807 19:49:49.671340 12505 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:49.716745 8783 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260807 19:49:49.703733 12500 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
I20260807 19:49:49.723177 9019 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:49.723829 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:49.726984 9027 raft_consensus.cc:2410] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260807 19:49:49.724197 12510 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.095s user 0.001s sys 0.000s
W20260807 19:49:49.725159 12508 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.106s user 0.000s sys 0.000s
I20260807 19:49:49.732574 9019 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260807 19:49:49.727551 12416 consensus_meta.cc:287] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.690s user 0.013s sys 0.000s
I20260807 19:49:49.744239 9027 raft_consensus.cc:1275] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:49.759955 8915 raft_consensus.cc:2410] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:49:49.735245 12084 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:49.760761 12084 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.762770 12084 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:49.764756 12512 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
W20260807 19:49:49.768596 12476 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.729s user 0.020s sys 0.012s
I20260807 19:49:49.770612 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:49.774186 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1681) took 1013 ms. Trace:
I20260807 19:49:49.774614 8789 rpcz_store.cc:276] 0807 19:49:48.760401 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:48.760664 (+ 263us) service_pool.cc:224] Handling call
0807 19:49:49.774017 (+1013353us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540009f954. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:49.774121 (+ 104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956488448}
I20260807 19:49:49.692605 12514 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.777339 8779 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:49.780072 8904 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260807 19:49:49.782423 8904 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:49.784129 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:49.815562 8904 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:49.815973 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:49.816591 9008 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:49.792918 9018 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:49.819160 8904 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:49.820741 8904 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:49.827194 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:49.828027 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:49.828118 9018 raft_consensus.cc:2393] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:49.828873 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:49.830163 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:49.837513 8904 raft_consensus.cc:1275] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260807 19:49:49.838011 8904 pending_rounds.cc:85] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 4
I20260807 19:49:49.838246 8904 pending_rounds.cc:107] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260807 19:49:49.838482 8904 raft_consensus.cc:2889] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:49.840029 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:49.841895 9008 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:49.442370 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:49.851033 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:49.799021 12382 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:49.864749 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1098) took 1478 ms. Trace:
I20260807 19:49:49.842571 12152 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:49.865283 8913 rpcz_store.cc:276] 0807 19:49:48.386527 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:48.387210 (+ 683us) service_pool.cc:224] Handling call
0807 19:49:48.387552 (+ 342us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:48.594768 (+207216us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:48.594802 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:48.606130 (+ 11328us) log.cc:844] Serialized 26 byte log entry
0807 19:49:48.759454 (+153324us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:48.810250 (+ 50796us) log.cc:844] Serialized 14 byte log entry
0807 19:49:48.814684 (+ 4434us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:48.814765 (+ 81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:49.767841 (+953076us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:49.862153 (+ 94312us) raft_consensus.cc:1685] finished
0807 19:49:49.864525 (+ 2372us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:49.864673 (+ 148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":872,"prepare.queue_time_us":49932,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":194734,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":244,"spinlock_wait_cycles":11847552,"thread_start_us":329519,"threads_started":2,"wal-append.queue_time_us":362148}
I20260807 19:49:49.873266 12152 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:49.876965 12462 consensus_queue.cc:1059] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260807 19:49:49.877866 12462 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:49.878229 12462 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:49.878372 12152 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.879145 12462 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:49.876433 12511 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.051s user 0.001s sys 0.000s
I20260807 19:49:49.894030 8856 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:49.897635 8913 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:49.899169 8913 raft_consensus.cc:1275] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260807 19:49:49.899513 8913 pending_rounds.cc:85] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 3
I20260807 19:49:49.899730 8913 pending_rounds.cc:107] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260807 19:49:49.899966 8913 raft_consensus.cc:2889] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:49.901331 11461 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:49.901897 11461 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:49.894793 12520 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.157s user 0.001s sys 0.000s
I20260807 19:49:49.867831 12382 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.910940 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:49.911808 9027 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:49.894156 12451 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:49.912889 8913 raft_consensus.cc:1275] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:49:49.923507 9027 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:49.924885 8856 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:49.925879 12425 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:49.926465 12425 raft_consensus.cc:697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:49.931799 11461 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.940997 12102 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:49:49.942714 12518 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.143s user 0.000s sys 0.001s
I20260807 19:49:49.927749 12425 consensus_queue.cc:237] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.930693 12433 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.952255 12433 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:49.956316 8911 raft_consensus.cc:1275] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:49.958032 8911 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260807 19:49:49.969540 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 93295.4us
I20260807 19:49:49.975935 8908 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:49.984652 8913 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:49.990993 12102 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:49.997376 12102 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.005472 12102 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:49.998425 12514 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.012418 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:50.014614 12505 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:50.012284 12433 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.032336 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:50.033170 8906 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:50.027217 12382 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.045025 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:50.045789 8908 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:50.046810 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:50.049335 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 69) took 3978 ms. Trace:
I20260807 19:49:50.053097 8906 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:50.054198 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:50.051715 8629 rpcz_store.cc:276] 0807 19:49:46.110743 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:46.117732 (+ 6989us) service_pool.cc:224] Handling call
0807 19:49:50.049222 (+3931490us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:48.116493 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:48.117091 (+ 598us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:48.117137 (+ 46us) write_op.cc:435] Acquired schema lock
0807 19:49:48.117155 (+ 18us) tablet.cc:662] Decoding operations
0807 19:49:48.117588 (+ 433us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:48.117646 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:48.117682 (+ 36us) tablet.cc:685] Acquiring locks for 2 operations
0807 19:49:48.118020 (+ 338us) tablet.cc:701] Row locks acquired
0807 19:49:48.118046 (+ 26us) write_op.cc:260] PREPARE: finished
0807 19:49:48.118200 (+ 154us) write_op.cc:270] Start()
0807 19:49:48.118339 (+ 139us) write_op.cc:276] Timestamp: P: 1786132188118151 usec, L: 0
0807 19:49:48.118359 (+ 20us) op_driver.cc:348] REPLICATION: starting
0807 19:49:48.119182 (+ 823us) log.cc:844] Serialized 974 byte log entry
0807 19:49:48.587579 (+468397us) op_driver.cc:464] REPLICATION: finished
0807 19:49:49.040980 (+453401us) write_op.cc:301] APPLY: starting
0807 19:49:49.047359 (+ 6379us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:49.077740 (+ 30381us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:49.091321 (+ 13581us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:49.121121 (+ 29800us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:49.121320 (+ 199us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 19:49:49.121364 (+ 44us) write_op.cc:312] APPLY: finished
0807 19:49:49.121618 (+ 254us) log.cc:844] Serialized 28 byte log entry
0807 19:49:49.141743 (+ 20125us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:49.147749 (+ 6006us) write_op.cc:454] Released schema lock
0807 19:49:49.149873 (+ 2124us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:49.241250 (+ 91377us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:49.243151 (+ 1901us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":40116,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":443129,"apply.run_cpu_time_us":13162,"apply.run_wall_time_us":188457,"num_ops":2,"prepare.queue_time_us":270546,"prepare.run_cpu_time_us":3142,"prepare.run_wall_time_us":9214,"replication_time_us":469041,"thread_start_us":505466,"threads_started":2}]]}
I20260807 19:49:50.063705 8972 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:50.092411 8972 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:50.093473 10988 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:50.094048 10988 raft_consensus.cc:697] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.095343 8972 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:50.098512 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:50.112722 10988 consensus_queue.cc:237] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.121421 8904 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260807 19:49:50.121892 8904 pending_rounds.cc:85] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 4
I20260807 19:49:50.122140 8904 pending_rounds.cc:107] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260807 19:49:50.122406 8904 raft_consensus.cc:2889] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:50.110175 12425 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.101908 12346 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
W20260807 19:49:50.149436 12544 log.cc:927] Time spent T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.087s user 0.001s sys 0.000s
I20260807 19:49:50.148972 12425 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.102475 12103 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 19:49:50.157761 8780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51024 (request call id 1684) took 1300 ms (client timeout 1558 ms). Trace:
I20260807 19:49:50.161715 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:50.162290 8776 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
W20260807 19:49:50.162943 8780 rpcz_store.cc:269] 0807 19:49:48.820090 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:48.820267 (+ 177us) service_pool.cc:224] Handling call
0807 19:49:50.120376 (+1300109us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540040b0d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:50.120517 (+ 141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2387884928}
I20260807 19:49:50.115813 12100 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 19:49:50.182852 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 84431.4us
I20260807 19:49:50.189147 12425 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.191241 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:50.167256 12492 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.221922 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.232935 12440 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:50.234303 12551 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:50.219959 12549 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260807 19:49:50.235936 12549 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.236989 12440 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.237620 12440 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.241123 12100 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:50.240926 12549 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.241847 12100 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.244105 12549 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.244921 12549 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:50.245286 12549 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.248469 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:50.252128 12542 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.069s user 0.000s sys 0.001s
I20260807 19:49:50.246203 12549 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.197484 12492 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:50.256402 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 63608.1us
I20260807 19:49:50.250111 8780 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:50.240226 12440 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.244146 12100 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.267020 8968 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:49:50.269065 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 65230.5us
W20260807 19:49:50.311424 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 44496.2us
I20260807 19:49:50.315191 8968 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:49:50.340991 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:50.291133 12550 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
W20260807 19:49:50.391251 8903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 1103) took 1813 ms (client timeout 1879 ms). Trace:
W20260807 19:49:50.391767 8903 rpcz_store.cc:269] 0807 19:49:48.578340 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:48.578494 (+ 154us) service_pool.cc:224] Handling call
0807 19:49:50.391037 (+1812543us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54002c3654. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:50.391182 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":977936256}
W20260807 19:49:50.393263 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 78187.2us
W20260807 19:49:50.392724 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 78510.2us
I20260807 19:49:50.387153 12556 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260807 19:49:50.388039 12550 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.396103 12100 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:50.396960 12564 consensus_queue.cc:1048] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260807 19:49:50.400748 12100 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:50.404134 10988 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:50.401292 12100 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:50.392778 12563 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260807 19:49:50.402035 12492 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.406840 10988 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.417151 10988 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.406488 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1451) took 1015 ms. Trace:
I20260807 19:49:50.421623 8904 rpcz_store.cc:276] 0807 19:49:49.390597 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:49.846016 (+455419us) service_pool.cc:224] Handling call
0807 19:49:50.121007 (+274991us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:50.122876 (+ 1869us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:50.406172 (+283296us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54001aaed0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:50.406415 (+ 243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":631644416}
I20260807 19:49:50.414608 12492 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.421468 12438 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:50.433540 12461 consensus_meta.cc:287] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.713s user 0.013s sys 0.000s
I20260807 19:49:50.409077 12556 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.029s
W20260807 19:49:50.457824 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 62865.6us
I20260807 19:49:50.425369 12100 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.444283 12438 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.465565 12438 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.469316 12100 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.470083 12100 consensus_queue.cc:1048] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260807 19:49:50.441655 12492 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.492051 12385 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.500115 11461 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.500309 12418 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:50.500855 12418 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.500819 11461 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.501863 12418 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.502913 12418 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:50.503189 11461 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.455173 12550 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.505246 12550 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:50.445454 12461 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:50.511240 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 104185us
W20260807 19:49:50.523763 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:50.524020 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:50.533161 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 72873us
I20260807 19:49:50.533896 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:50.536921 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:50.537771 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.540504 11461 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.543047 12565 consensus_queue.cc:1048] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.059s
I20260807 19:49:50.541081 11461 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.550318 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.545187 12388 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:50.566457 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:50.567308 9898 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:50.568578 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.569303 8779 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:50.570470 11461 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.603093 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:50.604336 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 70501.8us
I20260807 19:49:50.565066 12388 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:50.605676 8972 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:50.569766 12461 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:50.617178 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:50.617911 8910 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:49:50.618498 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 75976.4us
I20260807 19:49:50.623472 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:50.624230 8902 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:50.626123 8906 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260807 19:49:50.621810 8972 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:50.617923 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:50.630594 8968 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:50.630779 8904 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:50.634653 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:50.636590 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:50.615204 12388 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:50.640079 10988 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:50.640641 10988 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:50.641132 10988 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:50.648666 12564 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:50.649289 12564 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:50.649677 12564 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:50.652899 12581 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:50.653755 12581 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:50.619359 12140 raft_consensus.cc:2804] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:50.655973 10988 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.673807 11461 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:50.674803 10988 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:50.674355 11461 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.668653 8902 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:50.676474 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:50.677162 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:50.677220 8911 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:50.654072 12574 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.080s user 0.000s sys 0.001s
I20260807 19:49:50.679435 10988 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:50.634425 12566 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
W20260807 19:49:50.680083 12572 log.cc:927] Time spent T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260807 19:49:50.681910 11461 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.654178 12581 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:50.679977 10988 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.685761 9898 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:49:50.690372 8853 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:50.700161 8911 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:50.704950 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.702718 10988 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:50.709307 8972 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:50.692474 12140 raft_consensus.cc:697] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.732574 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.801719 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:50.802670 8789 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:50.704654 12461 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:50.706941 12492 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:50.834280 12461 raft_consensus.cc:697] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:50.838585 11461 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:50.852010 11461 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:50.855386 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:50.814602 12110 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:50.813089 9015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1439) took 1031 ms. Trace:
I20260807 19:49:50.897856 9015 rpcz_store.cc:276] 0807 19:49:49.781555 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:49.781689 (+ 134us) service_pool.cc:224] Handling call
0807 19:49:49.818190 (+ 36501us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:49.936980 (+118790us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:49.937010 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:49.937052 (+ 42us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:49.937091 (+ 39us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:50.812739 (+875648us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540041e6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:50.812806 (+ 67us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:50.813002 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272648960}
W20260807 19:49:50.925765 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 295243us
I20260807 19:49:50.927096 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.951478 11461 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:50.952373 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:50.835741 12461 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:50.957159 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 216122us
W20260807 19:49:50.937482 12536 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260807 19:49:50.961046 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.964871 8783 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:50.938108 12110 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:50.971854 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.972627 8783 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:49:50.974557 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 47961.8us
I20260807 19:49:50.975317 11461 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:50.975893 11461 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:50.978024 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.980708 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:50.989908 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:50.991364 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:50.993528 12438 consensus_queue.cc:1059] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260807 19:49:50.994668 12438 raft_consensus.cc:3055] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:50.995047 12438 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:50.997654 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.000830 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.005677 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.013208 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.014276 8775 raft_consensus.cc:2410] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:49:51.021377 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.022277 8788 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:51.023166 12586 log.cc:927] Time spent T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260807 19:49:51.024075 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:51.054891 12545 log.cc:927] Time spent T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.906s user 0.008s sys 0.014s
W20260807 19:49:51.063616 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:51.071568 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.091681 12110 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:51.093907 12557 log.cc:927] Time spent T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.651s user 0.004s sys 0.006s
I20260807 19:49:50.968803 12140 consensus_queue.cc:237] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.100368 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.102802 8780 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:51.117401 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.120440 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.121451 8776 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:51.122800 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.123587 8775 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 8
W20260807 19:49:51.124122 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 117273us
I20260807 19:49:51.133024 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.140105 11461 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.102948 12462 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:51.098075 12564 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.153002 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.156262 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.157048 9898 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:51.159060 12415 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:51.159648 12415 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.133886 12581 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.153776 8788 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:51.120522 12416 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.164222 8783 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260807 19:49:51.171940 8789 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:51.183138 8780 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:51.195417 8780 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:51.239302 12415 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.242604 12609 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.263670 12609 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.266187 12609 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.266824 12609 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.267263 12609 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.269923 12609 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:51.247761 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 81065us
I20260807 19:49:51.277570 11461 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:51.278206 11461 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.282051 11461 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.283509 11461 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.284052 11461 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.309921 12582 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:51.310601 12582 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:51.311789 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 102847us
I20260807 19:49:51.313181 8968 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:51.313349 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.314162 8902 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:51.315699 11461 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.319077 8775 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:49:51.319280 8788 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
W20260807 19:49:51.322306 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 184797us
W20260807 19:49:51.322522 12421 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.306s user 0.017s sys 0.026s
I20260807 19:49:51.172749 12416 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.323534 8789 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260807 19:49:51.322983 12421 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.306s user 0.017s sys 0.026s
I20260807 19:49:51.327385 11461 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:51.328107 11461 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.339291 12582 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.340317 12581 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.343430 11461 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.346081 12416 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.312616 12490 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
W20260807 19:49:51.438023 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:51.462399 11461 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:51.522497 11461 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:51.523609 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 188190us
W20260807 19:49:51.529654 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:51.444757 12550 consensus_meta.cc:287] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.939s user 0.012s sys 0.003s
I20260807 19:49:51.326159 12551 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:51.585481 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.531507 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:51.582525 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 149090us
I20260807 19:49:51.586591 8910 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:51.456048 8632 catalog_manager.cc:5697] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 6, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 6 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
W20260807 19:49:51.366434 12346 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.216s user 0.008s sys 0.031s
W20260807 19:49:51.587560 12346 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.438s user 0.022s sys 0.050s
I20260807 19:49:51.587901 12346 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:51.392982 12643 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.593379 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:51.593145 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 68784.5us
I20260807 19:49:51.594862 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.597195 8903 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:51.541481 12461 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.599560 11461 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.449549 12564 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.492024 12416 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:51.607046 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:49:51.607877 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 216308us
I20260807 19:49:51.610438 8913 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:51.544183 12538 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:51.612376 12385 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.624755 12550 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.625047 12103 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.629582 8787 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:51.647133 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.630626 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.632694 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.633440 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.634140 12466 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.400458 12621 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.203s
W20260807 19:49:51.636445 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 301818us
I20260807 19:49:51.664472 8787 raft_consensus.cc:1275] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:51.637596 12418 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.637041 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.335361 12577 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.270s
I20260807 19:49:51.592334 12551 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.638943 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1474) took 1013 ms. Trace:
W20260807 19:49:51.641729 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 318627us
I20260807 19:49:51.589066 12346 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.599782 8632 catalog_manager.cc:5697] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.685837 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:51.686362 10988 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.606609 12461 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.648871 12609 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.649113 12385 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.601557 12643 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.688760 12656 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:51.671104 12388 consensus_meta.cc:287] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.996s user 0.013s sys 0.000s
I20260807 19:49:51.698654 8913 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:51.719720 8910 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:51.649375 12571 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.696610 10988 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.709296 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.710505 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.658552 12103 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.712081 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:51.654330 12538 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.698726 8632 catalog_manager.cc:5697] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.638334 12416 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.697602 12551 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:51.721855 9017 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:51.723572 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.723485 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.658823 12550 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.696930 12614 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.726024 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:51.755187 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.711592 8911 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:51.733928 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.734107 8904 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:51.742756 8632 catalog_manager.cc:5697] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 3, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 3 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.691661 8906 rpcz_store.cc:276] 0807 19:49:50.625760 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:50.627239 (+ 1479us) service_pool.cc:224] Handling call
0807 19:49:50.627616 (+ 377us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:49:50.639199 (+ 11583us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:50.713193 (+ 73994us) log.cc:844] Serialized 14 byte log entry
0807 19:49:50.974032 (+260839us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:49:50.974667 (+ 635us) log.cc:844] Serialized 52 byte log entry
0807 19:49:50.974822 (+ 155us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:50.978656 (+ 3834us) log.cc:844] Serialized 14 byte log entry
0807 19:49:50.979109 (+ 453us) log.cc:844] Serialized 14 byte log entry
0807 19:49:51.367368 (+388259us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b18000e0860. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fae05 00007f687b5fc440 00007f687b60eaf6 00007f687b61f667 00007f687b61f5e8
0807 19:49:51.610304 (+242936us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:51.638601 (+ 28297us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:51.638656 (+ 55us) raft_consensus.cc:1685] finished
0807 19:49:51.638673 (+ 17us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:51.638865 (+ 192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68877,"prepare.queue_time_us":668139,"prepare.run_cpu_time_us":177,"prepare.run_wall_time_us":4617,"raft.queue_time_us":15189,"raft.run_cpu_time_us":727,"raft.run_wall_time_us":58553,"spinlock_wait_cycles":12487808,"thread_start_us":495408,"threads_started":3,"wal-append.queue_time_us":276769}
I20260807 19:49:51.748392 8913 raft_consensus.cc:2393] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:49:51.751772 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:51.705456 12102 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.720252 12382 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:51.764997 8632 catalog_manager.cc:5697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 5 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:51.729506 12140 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:51.773226 10988 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.723486 12490 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.769439 8906 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:51.708735 12609 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.770117 8902 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 2.
I20260807 19:49:51.661063 12466 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.720142 12631 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:51.775990 8776 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:51.777601 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.791862 9022 raft_consensus.cc:2393] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:51.781030 8906 raft_consensus.cc:1275] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:49:51.701455 12418 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.781773 12103 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.800879 8903 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:51.784359 12490 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.725762 12388 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.796933 8968 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:51.772265 8632 catalog_manager.cc:5697] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 4, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 4 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:51.772991 12140 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:51.801793 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 79759.7us
I20260807 19:49:51.806342 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.807220 8906 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260807 19:49:51.810289 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 83697us
I20260807 19:49:51.811996 8632 catalog_manager.cc:5697] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 5, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 5 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:51.770373 12614 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:51.814996 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 48240.2us
I20260807 19:49:51.816416 8632 catalog_manager.cc:5697] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.750425 12571 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:51.804916 12627 log.cc:927] Time spent T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.209s user 0.000s sys 0.001s
I20260807 19:49:51.826318 12571 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.825981 8632 catalog_manager.cc:5697] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:51.720312 12656 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.828004 12550 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.827730 8632 catalog_manager.cc:5697] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 5 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.829918 8632 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:51.831712 12577 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:51.832568 12622 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.769567 12102 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.833148 12622 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.826903 12571 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:49:51.805886 12631 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.835585 12577 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.861109 12491 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.871636 9898 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 19:49:51.808236 12140 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:51.833026 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.839551 12671 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.059s
I20260807 19:49:51.829918 12656 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.831740 8632 catalog_manager.cc:5697] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.880703 8968 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:51.775475 12382 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.861526 12637 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:51.861963 9889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.807379 12416 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.887549 12461 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.889989 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.890288 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.891301 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:51.892565 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.936511 12382 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.894942 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.897308 12622 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.898276 12102 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.869211 12385 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:51.899416 12624 log.cc:927] Time spent T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.581s user 0.004s sys 0.000s
I20260807 19:49:51.901777 12490 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.903889 12418 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.866752 12538 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.903600 12491 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.868194 12609 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.932652 12380 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:51.941507 12491 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.942619 12438 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.943699 12438 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:51.943945 9893 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.944749 12491 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 19:49:51.946089 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 54881.2us
I20260807 19:49:51.893326 12643 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:51.908977 9891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.910053 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.911078 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.933396 12637 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:51.913192 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.913996 12577 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.921861 12631 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:51.868820 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:51.953143 12643 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:51.953503 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.954139 9015 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:51.954645 12643 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:51.955220 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.956133 12643 consensus_queue.cc:1059] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260807 19:49:51.930332 8632 catalog_manager.cc:5697] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 4, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 4 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
W20260807 19:49:51.931525 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 50922.5us
I20260807 19:49:51.894533 12687 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:51.962306 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.963259 8632 catalog_manager.cc:5697] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 3 to 6, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.964607 9015 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:51.966297 12385 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:51.968201 12388 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:51.970286 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.971159 12380 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:51.971743 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.972481 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.973222 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.974052 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.974489 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.974638 8787 raft_consensus.cc:2376] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8. Re-sending same reply.
I20260807 19:49:51.975807 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.964864 8632 catalog_manager.cc:5697] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:51.976713 10988 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.527s
I20260807 19:49:52.406411 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1468) took 1784 ms. Trace:
W20260807 19:49:52.159890 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 274613us
W20260807 19:49:52.407441 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 590125us
I20260807 19:49:52.508839 8856 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.510170 8856 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.509235 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.511475 8856 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:51.979713 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.512775 11461 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:52.513207 11461 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:52.513640 11461 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:51.977316 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.040940 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.516019 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:51.980221 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.979280 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.518982 8783 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:52.520041 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:51.976975 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:51.976517 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.023514 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.528838 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.520808 9023 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:51.975859 12490 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.619s
I20260807 19:49:52.595273 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.596982 9023 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:52.597424 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.592518 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.598670 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.599042 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.601368 8778 raft_consensus.cc:3055] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:51.992928 12466 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.595952 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.602137 8778 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:52.602523 8780 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:52.037742 12638 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:52.594846 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.603065 8778 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.604782 8778 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:51.985714 12687 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:52.605722 12584 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 1.683s user 0.000s sys 0.019s
I20260807 19:49:52.533991 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.607028 12687 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.533547 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.607606 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1307) took 2085 ms. Trace:
W20260807 19:49:52.043040 12636 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.725s user 0.004s sys 0.000s
I20260807 19:49:52.603984 12638 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.609393 10988 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.609874 10988 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.610437 12638 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.611615 10988 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:52.612301 12638 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:52.612920 12638 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.535310 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:52.026582 12688 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
I20260807 19:49:52.609445 12466 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260807 19:49:52.615828 8778 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260807 19:49:52.616285 8778 pending_rounds.cc:85] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:52.616545 8778 pending_rounds.cc:107] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260807 19:49:52.405539 12462 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:49:52.616783 8778 raft_consensus.cc:2889] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:52.552134 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.607949 8790 rpcz_store.cc:276] 0807 19:49:50.522268 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:50.522648 (+ 380us) service_pool.cc:224] Handling call
0807 19:49:50.523083 (+ 435us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:50.533356 (+ 10273us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:50.533389 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:50.533860 (+ 471us) log.cc:844] Serialized 26 byte log entry
0807 19:49:50.670707 (+136847us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:51.206508 (+535801us) log.cc:844] Serialized 14 byte log entry
0807 19:49:51.516335 (+309827us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:51.516461 (+ 126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:52.605139 (+1088678us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:49:52.607300 (+ 2161us) raft_consensus.cc:1685] finished
0807 19:49:52.607421 (+ 121us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:52.607548 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12568,"prepare.queue_time_us":153836,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":157889,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":40302,"spinlock_wait_cycles":8420736,"thread_start_us":875711,"threads_started":3,"wal-append.queue_time_us":235008}
I20260807 19:49:52.619270 9018 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:52.507594 8901 rpcz_store.cc:276] 0807 19:49:50.622277 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:50.623002 (+ 725us) service_pool.cc:224] Handling call
0807 19:49:50.636985 (+ 13983us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:50.637392 (+ 407us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:51.460820 (+823428us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b18001716a0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fae05 00007f687f01bda7 00007f687b60ed87 00007f687b61e49f 00007f687b61e418
0807 19:49:51.717178 (+256358us) log.cc:844] Serialized 14 byte log entry
0807 19:49:51.797637 (+ 80459us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:51.798181 (+ 544us) log.cc:844] Serialized 26 byte log entry
0807 19:49:51.798319 (+ 138us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:51.811512 (+ 13193us) log.cc:844] Serialized 14 byte log entry
0807 19:49:52.051052 (+239540us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b18001716a0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f6878477d1a 00007f6881c0b5d9 00007f68785003a0 00007f68784fae94 00007f68784fae05 00007f687b5fc440 00007f687b60eaf6 00007f687b61f667 00007f687b61f5e8
0807 19:49:52.404965 (+353913us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:52.405071 (+ 106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:52.405130 (+ 59us) raft_consensus.cc:1685] finished
0807 19:49:52.405142 (+ 12us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:52.406313 (+ 1171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":254027,"prepare.queue_time_us":333925,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":6287,"raft.queue_time_us":277212,"raft.run_cpu_time_us":485,"raft.run_wall_time_us":77504,"spinlock_wait_cycles":12176128,"thread_start_us":655429,"threads_started":3,"wal-append.queue_time_us":71341}
I20260807 19:49:52.593951 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:52.517215 12416 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:52.031723 12637 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:52.621371 12638 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.622570 12614 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:52.650213 11461 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.517414 12461 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:52.532886 12442 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:49:52.639585 12346 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s user 0.035s sys 0.066s
I20260807 19:49:51.991490 12551 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:51.992043 12438 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.660s
W20260807 19:49:52.655045 12346 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s user 0.035s sys 0.067s
I20260807 19:49:52.276548 12662 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.404s
I20260807 19:49:52.666596 11461 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.668742 9020 raft_consensus.cc:1240] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:52.655890 12549 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:49:52.670440 8853 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.025s sys 0.017s
W20260807 19:49:52.671525 8853 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s user 0.025s sys 0.017s
I20260807 19:49:52.663552 12637 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:52.660964 12442 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:52.662461 12638 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:52.669482 12466 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260807 19:49:52.680871 12694 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.674616 12637 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.705917 11461 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:52.706591 11461 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.678854 12638 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:52.718926 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:52.720525 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 188844us
I20260807 19:49:52.674971 12442 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:52.721771 11461 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.702764 12438 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:52.724702 9838 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.735663 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.745946 12415 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:52.746366 12637 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.746737 12415 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:49:52.747689 12689 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
W20260807 19:49:52.750666 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 200025us
I20260807 19:49:52.704286 12694 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.753242 8856 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:52.759819 11461 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:52.760233 11461 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:52.760288 12581 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.760670 11461 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:52.761016 12581 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.763383 12581 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:52.765120 12691 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.084s user 0.001s sys 0.000s
W20260807 19:49:52.766569 12690 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260807 19:49:52.735797 12438 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.770932 12438 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.776690 12612 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.777310 12612 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.779456 12551 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:52.779837 12462 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:52.780473 12462 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:52.792881 8856 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:52.794371 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:52.750365 12698 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260807 19:49:52.797919 12698 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:52.798346 12698 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:52.798686 12698 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:52.801579 11461 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.805012 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.805698 8775 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:52.806571 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.807569 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.808248 8787 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:52.807425 11461 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.810784 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.819963 12694 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.777364 12638 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.820008 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.821297 8783 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
W20260807 19:49:52.823382 12140 consensus_meta.cc:287] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.892s user 0.006s sys 0.007s
I20260807 19:49:52.823819 12140 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.825043 9022 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:52.826570 8775 raft_consensus.cc:2410] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260807 19:49:52.833956 12551 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.791635 12462 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:52.841027 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.795265 12612 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:52.845969 8775 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:52.848961 8908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1472) took 2224 ms. Trace:
I20260807 19:49:52.849505 8908 rpcz_store.cc:276] 0807 19:49:50.624832 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:50.626357 (+ 1525us) service_pool.cc:224] Handling call
0807 19:49:50.655352 (+ 28995us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:50.664711 (+ 9359us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:50.664748 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:50.673836 (+ 9088us) log.cc:844] Serialized 26 byte log entry
0807 19:49:52.533060 (+1859224us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b1800332ec0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a32c1 00007f687b5bfdef
0807 19:49:52.736697 (+203637us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:52.836091 (+ 99394us) log.cc:844] Serialized 14 byte log entry
0807 19:49:52.845187 (+ 9096us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:52.845273 (+ 86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:52.845334 (+ 61us) raft_consensus.cc:1685] finished
0807 19:49:52.848706 (+ 3372us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:52.848895 (+ 189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9619,"raft.queue_time_us":17884,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":10679,"spinlock_wait_cycles":10824576,"thread_start_us":302401,"threads_started":2,"wal-append.queue_time_us":2084771}
I20260807 19:49:52.803948 12438 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:52.853322 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.854403 9015 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:52.859169 8783 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 19:49:52.860082 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.861541 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.855024 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.862862 9023 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:52.856215 9008 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:52.864135 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.865712 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.867056 9008 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:52.868968 9018 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:52.870075 9017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1466) took 1130 ms. Trace:
W20260807 19:49:52.870430 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 127343us
I20260807 19:49:52.873521 9018 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 2. Current term is 3. Ops: []
I20260807 19:49:52.875036 9018 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:49:52.870517 9017 rpcz_store.cc:276] 0807 19:49:51.721930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.766361 (+ 44431us) service_pool.cc:224] Handling call
0807 19:49:51.771241 (+ 4880us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:51.937168 (+165927us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:51.937198 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:51.937237 (+ 39us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:51.937266 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:52.852345 (+915079us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000d43d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:52.852415 (+ 70us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:52.852549 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381133056}
I20260807 19:49:52.880148 9023 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:52.843802 12575 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.884207 9017 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:52.881199 12728 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:52.887739 12728 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:52.889034 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1479) took 1358 ms. Trace:
I20260807 19:49:52.847278 12140 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:52.892117 9021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1462) took 1172 ms. Trace:
I20260807 19:49:52.829535 12442 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.889438 8902 rpcz_store.cc:276] 0807 19:49:51.530589 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.531181 (+ 592us) service_pool.cc:224] Handling call
0807 19:49:52.888855 (+1357674us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b54002fbf54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:52.888971 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422019968}
I20260807 19:49:52.895955 8775 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:52.892587 9021 rpcz_store.cc:276] 0807 19:49:51.719755 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.722769 (+ 3014us) service_pool.cc:224] Handling call
0807 19:49:51.723062 (+ 293us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:51.727370 (+ 4308us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:51.727400 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:51.727442 (+ 42us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:49:51.727470 (+ 28us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:52.891846 (+1164376us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b5400449654. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:52.891902 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:52.892047 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9435392}
W20260807 19:49:52.900565 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:52.834585 12551 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.904127 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.904979 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.900286 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:52.907997 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 115215us
I20260807 19:49:52.911450 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.912629 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.913923 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.914256 8856 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.914784 9021 raft_consensus.cc:2410] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
W20260807 19:49:52.915014 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 173012us
W20260807 19:49:52.916024 8849 leader_election.cc:336] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.604s (SENT)
I20260807 19:49:52.916657 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.917588 12694 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:52.918869 12694 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:52.919123 8849 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.919973 11461 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:52.919576 12694 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:52.922621 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.923450 11461 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:52.923483 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:49:52.924417 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:52.924943 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.925772 11461 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:52.925838 12694 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:52.927351 8856 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:52.927608 8849 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:52.927801 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.935425 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:52.936642 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.936864 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.937268 8780 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:52.938128 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.939081 11461 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:52.939448 12694 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:52.940819 12694 raft_consensus.cc:2764] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260807 19:49:52.941423 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:52.943003 9021 raft_consensus.cc:2393] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:52.939546 11461 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:52.944720 11461 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:52.946564 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37380 (request call id 228) took 1228 ms. Trace:
I20260807 19:49:52.946971 9019 rpcz_store.cc:276] 0807 19:49:51.717788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.717909 (+ 121us) service_pool.cc:224] Handling call
0807 19:49:51.718178 (+ 269us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:51.783003 (+ 64825us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:51.783033 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:51.783071 (+ 38us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:49:51.783100 (+ 29us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:52.946324 (+1163224us) spinlock_profiling.cc:243] Waited 64.6 ms on lock 0x7b54003f6e54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:52.946377 (+ 53us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:52.946501 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148625280}
I20260807 19:49:52.912179 12705 raft_consensus.cc:3055] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:52.949213 12705 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:52.951134 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.951387 12687 raft_consensus.cc:3055] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:52.951980 12687 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:52.952971 12687 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.897152 12538 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:52.934383 8968 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:52.878875 12438 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:52.954041 12687 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:52.910207 12637 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.931520 12581 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.970345 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.970196 11461 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.972601 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:52.979485 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.971987 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.980737 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:52.980834 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.932910 12710 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:52.964654 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.984714 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.974390 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:52.978520 8902 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:52.981489 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:52.981917 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.975445 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:52.987986 8789 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:52.982682 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.989013 8910 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:52.982602 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.979797 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.984146 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.893890 12575 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:52.985353 8904 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:52.991039 8915 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:52.991074 11461 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:52.993311 8915 raft_consensus.cc:1240] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:52.995617 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.996521 12552 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:52.997140 8789 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:52.997572 8789 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:52.998453 8968 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:52.998405 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:52.972646 12416 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.931843 12704 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260807 19:49:53.004441 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.007335 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:53.005650 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.975695 12556 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:53.008701 8910 raft_consensus.cc:2410] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260807 19:49:53.007465 8911 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
I20260807 19:49:53.008737 8968 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:53.004449 8904 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:53.001246 8789 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.013077 8789 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:53.035832 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.037604 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.038380 8913 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:52.973172 12538 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:52.997421 12140 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:52.974848 12699 consensus_queue.cc:1059] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260807 19:49:53.002252 12462 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.005869 12549 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:53.006924 12729 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:53.058735 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:52.982167 12637 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:53.061432 8913 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260807 19:49:53.009218 12556 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:52.985014 12581 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.079490 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.083226 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.084174 8911 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 8: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:53.072978 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.080955 9023 raft_consensus.cc:1240] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:53.104959 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 149) took 1221 ms. Trace:
I20260807 19:49:53.056723 12715 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:53.147197 12715 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:53.147591 12715 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.057941 12140 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:53.153268 12731 log.cc:927] Time spent T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
I20260807 19:49:53.062700 12549 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:53.153970 12549 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:53.005532 12679 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260807 19:49:53.048722 12466 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:53.060175 12609 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:53.005992 12552 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260807 19:49:53.105481 8788 rpcz_store.cc:276] 0807 19:49:51.888513 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.888649 (+ 136us) service_pool.cc:224] Handling call
0807 19:49:51.888986 (+ 337us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:51.997262 (+108276us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:51.997298 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:51.997344 (+ 46us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:51.997374 (+ 30us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:53.104679 (+1107305us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540008b6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:53.104742 (+ 63us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:53.104891 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248415616}
I20260807 19:49:53.172109 12730 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:53.072266 12556 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:53.199244 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:53.176431 12551 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.070492 12462 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.178038 12715 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.178685 12581 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.181244 9019 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:53.185437 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.014335 12416 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.190284 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.003170 12710 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:53.214504 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.218269 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.075994 9885 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33452 (request call id 269) took 1426 ms. Trace:
W20260807 19:49:53.220678 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 293118us
I20260807 19:49:53.221765 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.219075 9017 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:53.163661 12637 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:53.225503 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 216839us
W20260807 19:49:53.232527 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.192210 12609 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:53.233107 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.242372 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.243124 8779 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 3.
W20260807 19:49:53.248557 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 312898us
W20260807 19:49:53.220201 12748 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.062s user 0.000s sys 0.000s
I20260807 19:49:53.249939 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.248107 8906 raft_consensus.cc:2376] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5. Re-sending same reply.
I20260807 19:49:53.251173 8911 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:53.253652 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.257210 9017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:53.257838 9017 raft_consensus.cc:606] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:53.258253 9017 raft_consensus.cc:614] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:53.202452 12466 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.268875 12749 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:53.269642 12749 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:53.271770 12715 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:53.260241 12733 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.181s user 0.001s sys 0.000s
W20260807 19:49:53.272897 12722 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.228s user 0.000s sys 0.001s
I20260807 19:49:53.220010 12730 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:53.241519 12442 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.247697 12609 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.076653 12729 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.285732 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.226861 9885 rpcz_store.cc:276] 0807 19:49:51.634975 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.635183 (+ 208us) service_pool.cc:224] Handling call
0807 19:49:51.635565 (+ 382us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:51.782404 (+146839us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:51.782456 (+ 52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:51.782505 (+ 49us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:49:51.782538 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:53.039399 (+1256861us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54000e33d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:53.043802 (+ 4403us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:53.059576 (+ 15774us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313196288}
I20260807 19:49:53.286340 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.303771 12575 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.308277 12538 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.285847 12438 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:53.350593 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:53.351450 9027 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37380 (request call id 231) took 1401 ms (client timeout 1853 ms). Trace:
W20260807 19:49:53.351858 9027 rpcz_store.cc:269] 0807 19:49:51.926727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.959878 (+ 33151us) service_pool.cc:224] Handling call
0807 19:49:53.286141 (+1326263us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:49:53.328224 (+ 42083us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5404032}
W20260807 19:49:53.354592 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 132861us
I20260807 19:49:53.352985 12462 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.358930 8849 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.374981 12729 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.370014 12582 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:53.385433 12552 consensus_queue.cc:1048] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:53.386637 12710 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:53.393127 12416 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.396849 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.368487 12549 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.411147 12549 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.412245 9021 raft_consensus.cc:1240] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:53.414340 12416 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:53.414979 12416 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.416493 12442 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:53.417327 12416 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:53.382253 12698 consensus_meta.cc:287] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.583s user 0.011s sys 0.002s
I20260807 19:49:53.418041 12698 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.420212 8775 raft_consensus.cc:3055] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:53.420667 8775 raft_consensus.cc:740] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:53.421483 8775 consensus_queue.cc:260] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.422724 8775 raft_consensus.cc:3060] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:53.399991 9850 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.429369 9022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57870 (request call id 1471) took 1635 ms. Trace:
I20260807 19:49:53.430042 9020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1465) took 1708 ms. Trace:
I20260807 19:49:53.430505 9020 rpcz_store.cc:276] 0807 19:49:51.721669 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.722134 (+ 465us) service_pool.cc:224] Handling call
0807 19:49:51.722433 (+ 299us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:52.669202 (+946769us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:53.429735 (+760533us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540043c1d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:49:53.429964 (+ 229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2175863040}
I20260807 19:49:53.429440 12422 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:53.436614 12749 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:53.437480 12749 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:53.438035 12749 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:53.441604 8904 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260807 19:49:53.442679 8856 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.195:35801 timed out after 2.622s (SENT) [suppressed 20 similar messages]
W20260807 19:49:53.429150 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 70322.3us
I20260807 19:49:53.443634 8849 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.417130 12442 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.445981 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.429854 9022 rpcz_store.cc:276] 0807 19:49:51.793657 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:51.795415 (+ 1758us) service_pool.cc:224] Handling call
0807 19:49:53.429134 (+1633719us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54000d48d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:53.429300 (+ 166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2336787712}
I20260807 19:49:53.448756 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:53.458779 9015 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260807 19:49:53.460677 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 64391.6us
I20260807 19:49:53.465190 8775 raft_consensus.cc:1275] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260807 19:49:53.465741 8775 pending_rounds.cc:85] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:53.466113 8775 pending_rounds.cc:107] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260807 19:49:53.466516 8775 raft_consensus.cc:2889] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:53.468693 9017 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:53.420230 12698 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.455848 12582 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:53.475203 12662 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:53.477083 12466 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.475647 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.481065 12760 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.481654 12760 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.475629 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.487448 12556 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.489480 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.461799 12729 consensus_queue.cc:1059] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260807 19:49:53.492600 12729 raft_consensus.cc:3055] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:53.492974 12729 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:53.411376 12730 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:53.493860 12729 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:53.499631 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56082.6us
W20260807 19:49:53.513561 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 52257.6us
I20260807 19:49:53.513841 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.502381 12462 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:53.515075 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.506510 12705 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.507848 12442 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.508764 12609 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.530886 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:53.484192 12760 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.510406 12582 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.501008 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.475828 12662 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:53.547047 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:53.547475 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 46560.5us
I20260807 19:49:53.517781 9008 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:53.548799 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.519311 8849 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.525344 12612 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:53.551282 12346 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:53.526788 12681 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:53.527576 12728 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:53.556938 12438 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.557166 12728 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.557570 12438 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.557729 12728 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.559369 12699 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.528191 12462 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:53.559906 12728 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.528839 12750 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:53.556530 12612 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.562361 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.563082 9890 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:53.563508 12612 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.469894 12749 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:53.524016 12140 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:53.564448 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 06dc193f54374639a09c9b5827ff15f4 (from a38f4d6088cf4c8cb510200ca0555801 to 8e9ad2b5e2a94d4faff90af735c6d792) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:53.560112 12490 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:53.566188 12687 consensus_meta.cc:287] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.590s user 0.017s sys 0.008s
I20260807 19:49:53.566872 12729 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:53.569991 12609 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.561818 12422 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:53.559942 12699 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.553459 8849 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.556489 12346 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:53.576712 8737 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:53.581379 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.566205 12490 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.582777 12699 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.580524 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:53.566406 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.584216 9008 raft_consensus.cc:606] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:53.575024 12760 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:49:53.586921 12662 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:53.580698 12749 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:53.588014 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:53.577544 12346 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:53.589114 12749 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:53.589555 12749 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:53.589943 12749 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:53.590127 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:53.552062 12705 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.587124 12556 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.593497 12728 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:53.593943 12728 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.594305 12728 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.597040 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.583935 12490 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.588762 9008 raft_consensus.cc:606] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:53.598034 9008 raft_consensus.cc:614] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260807 19:49:53.599287 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 8666b7cb7a05408a8223317d32823bc9 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:53.559826 12438 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:53.600871 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.601383 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:53.601923 8849 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:53.604487 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 161825us
I20260807 19:49:53.607336 9008 raft_consensus.cc:606] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:53.610518 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.611368 9015 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 7: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:53.663548 12729 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:53.665350 8740 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:53.665776 8849 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.611358 12462 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:53.672623 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:53.726114 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
W20260807 19:49:53.750020 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 77490.9us
I20260807 19:49:53.750872 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:53.751181 12780 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:53.751703 12780 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.751711 12775 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:53.752174 12780 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:53.705678 12346 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.732549 12712 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:53.668563 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.752133 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:53.750365 9008 raft_consensus.cc:606] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
W20260807 19:49:53.748672 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:53.761247 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.761209 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.761979 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260807 19:49:53.762428 12775 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:53.762931 12775 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:53.762001 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.764712 12490 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:53.765105 12490 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:53.765027 12462 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.765738 12490 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:53.766440 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.765055 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:53.768702 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:53.769629 12738 consensus_queue.cc:1059] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:53.770524 12738 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:53.771190 12738 raft_consensus.cc:3055] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:53.771536 12738 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:53.768545 12637 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:53.772925 12637 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.772605 12738 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.773396 12637 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:53.774014 12738 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.765905 12462 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.777397 12462 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.703640 12438 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.076s
I20260807 19:49:53.779868 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.670526 12760 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:49:53.668613 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.728613 8968 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:53.781904 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:53.685972 12777 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:53.782542 8775 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.786985 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.787691 8787 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:53.756814 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.782251 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:49:53.702636 12782 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260807 19:49:53.790940 12687 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:49:53.791045 12699 raft_consensus.cc:3055] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
W20260807 19:49:53.781919 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 113451us
W20260807 19:49:53.791896 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 63401.2us
I20260807 19:49:53.791379 12687 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:53.792613 12687 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:53.792776 12681 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:49:53.793238 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.795149 12699 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:53.796082 12699 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.797346 12699 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:53.684716 12551 raft_consensus.cc:993] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:49:53.800730 12612 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:53.801136 12612 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:53.801509 12612 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:53.802325 12140 raft_consensus.cc:1081] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Signalling peer 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:49:53.804303 12140 consensus_queue.cc:1048] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:53.808316 12782 consensus_queue.cc:1059] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260807 19:49:53.809296 12709 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:53.809697 12709 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:53.810557 12709 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.811810 12709 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:53.816866 12705 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:53.818637 12705 raft_consensus.cc:3055] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:53.819311 12705 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:53.798170 12462 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:49:53.782747 12729 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260807 19:49:53.821648 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.829005 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:49:53.830077 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:53.830364 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
W20260807 19:49:53.830750 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.831926 8775 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:53.833209 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.835347 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.836000 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:53.755144 12346 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:53.822580 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.705811 12728 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:53.725272 8629 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 4 to 6, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 6 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:49:53.842190 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.688442 12749 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.844091 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.827203 12729 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:53.849292 11461 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:53.850355 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.854015 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:53.854982 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.831827 12782 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260807 19:49:53.858582 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:53.861500 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.862214 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.864996 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:53.864900 9015 raft_consensus.cc:2393] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:53.862535 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.865643 9021 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:53.866634 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.867838 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.868918 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.869269 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.869661 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.870256 8904 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:53.870564 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.871606 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.872191 8910 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:49:53.872988 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.873104 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.873803 9019 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:53.873713 8911 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:53.876559 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.871207 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:53.877738 8901 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:53.836500 12775 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.869879 8913 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:49:53.881322 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.826797 12712 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.884089 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.897873 9021 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:49:53.898959 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.850914 12777 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:53.899644 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.903400 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.848855 12780 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.853636 12729 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:53.910744 8783 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:53.916682 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:53.917452 8776 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:49:53.923111 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56572.9us
I20260807 19:49:53.843204 12662 consensus_queue.cc:1059] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260807 19:49:53.910457 12780 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:53.850561 12462 consensus_queue.cc:1059] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260807 19:49:53.856949 12730 consensus_queue.cc:1059] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260807 19:49:53.905786 12438 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:53.903815 12759 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:53.905628 12775 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:53.920046 12442 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:53.954421 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:53.891089 12638 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260807 19:49:53.956468 8856 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.938519 12704 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:53.964259 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:53.936326 12777 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:53.966367 8632 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38426 (request call id 208) took 6125 ms. Trace:
I20260807 19:49:53.864040 12705 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.987600 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:53.949493 12728 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:54.013118 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 131879us
I20260807 19:49:54.015538 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.976693 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:53.991796 12775 raft_consensus.cc:3055] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:53.998420 12681 raft_consensus.cc:3055] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:54.033171 8856 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.034996 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:53.966778 8632 rpcz_store.cc:276] 0807 19:49:47.836833 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:47.837081 (+ 248us) service_pool.cc:224] Handling call
0807 19:49:53.961810 (+6124729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:52.928204 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:53.104300 (+176096us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:53.155275 (+ 50975us) write_op.cc:435] Acquired schema lock
0807 19:49:53.155298 (+ 23us) tablet.cc:662] Decoding operations
0807 19:49:53.170955 (+ 15657us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:53.171057 (+ 102us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:53.171085 (+ 28us) tablet.cc:685] Acquiring locks for 15 operations
0807 19:49:53.172311 (+ 1226us) tablet.cc:701] Row locks acquired
0807 19:49:53.172329 (+ 18us) write_op.cc:260] PREPARE: finished
0807 19:49:53.172530 (+ 201us) write_op.cc:270] Start()
0807 19:49:53.172663 (+ 133us) write_op.cc:276] Timestamp: P: 1786132193172482 usec, L: 0
0807 19:49:53.172685 (+ 22us) op_driver.cc:348] REPLICATION: starting
0807 19:49:53.173548 (+ 863us) log.cc:844] Serialized 5928 byte log entry
0807 19:49:53.462906 (+289358us) op_driver.cc:464] REPLICATION: finished
0807 19:49:53.690601 (+227695us) write_op.cc:301] APPLY: starting
0807 19:49:53.690691 (+ 90us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:53.690994 (+ 303us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:53.691109 (+ 115us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:53.704899 (+ 13790us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:53.705170 (+ 271us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0807 19:49:53.705214 (+ 44us) write_op.cc:312] APPLY: finished
0807 19:49:53.705582 (+ 368us) log.cc:844] Serialized 106 byte log entry
0807 19:49:53.712662 (+ 7080us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:53.712842 (+ 180us) write_op.cc:454] Released schema lock
0807 19:49:53.712900 (+ 58us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:53.713320 (+ 420us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:53.713349 (+ 29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5291,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":227439,"apply.run_cpu_time_us":4001,"apply.run_wall_time_us":22677,"num_ops":15,"prepare.queue_time_us":276577,"prepare.run_cpu_time_us":19627,"prepare.run_wall_time_us":463895,"raft.queue_time_us":84149,"raft.run_cpu_time_us":92080,"raft.run_wall_time_us":118303,"replication_time_us":290091,"thread_start_us":619464,"threads_started":4,"wal-append.queue_time_us":202342}]]}
I20260807 19:49:53.960852 12759 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:53.986147 12612 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.042796 8856 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:54.095901 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 53190.2us
I20260807 19:49:54.052819 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:53.996627 12766 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260807 19:49:54.101069 8775 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:54.022869 12775 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:54.096987 8856 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:54.106601 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.107471 8901 raft_consensus.cc:2393] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:54.108071 9018 raft_consensus.cc:1240] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
W20260807 19:49:54.108107 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.109364 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.110240 8790 pending_rounds.cc:85] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:54.112290 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.113938 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.030184 12442 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:54.019528 12728 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:54.022428 12761 raft_consensus.cc:3055] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:54.039203 12730 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:54.031008 12681 raft_consensus.cc:740] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:54.032104 12705 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:54.102909 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.134327 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.097237 8776 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:54.105830 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.110628 8790 pending_rounds.cc:107] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260807 19:49:54.043784 12462 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:54.125811 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.097616 12767 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:54.131738 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.150709 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.106030 8789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:51024
I20260807 19:49:54.151465 8789 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:49:54.151966 8789 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:54.136823 8790 raft_consensus.cc:2889] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:54.140019 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.145335 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.157699 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.125495 12749 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:54.168960 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.133194 12694 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:54.131794 12761 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:54.182377 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.188753 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:54.197099 8775 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:54.143481 12438 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:54.150473 12712 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:54.042881 12777 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.208053 8904 raft_consensus.cc:1275] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260807 19:49:54.203215 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 106469us
I20260807 19:49:54.209785 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:54.125335 12789 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260807 19:49:54.171278 12728 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:54.220399 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.175941 12782 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:54.224471 8904 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:54.227345 8904 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:54.229907 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:54.231215 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 62536.3us
I20260807 19:49:54.232510 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.234342 8904 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:49:54.219038 12694 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:49:54.230468 12787 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.102s user 0.000s sys 0.001s
I20260807 19:49:54.224807 12614 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:49:54.206981 12637 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.251356 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:54.257479 9022 raft_consensus.cc:2368] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
W20260807 19:49:54.252086 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.212110 12687 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.253119 9008 raft_consensus.cc:1240] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:54.233577 12438 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:54.263294 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.265156 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:54.259040 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 73677.7us
I20260807 19:49:54.252940 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.176319 12775 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.271389 8856 leader_election.cc:400] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:54.269073 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.271831 8856 leader_election.cc:403] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260807 19:49:54.272541 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.269213 8904 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:54.266112 8902 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:54.259977 12614 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:54.280216 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.262156 12140 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:54.282074 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.246466 12767 consensus_queue.cc:1059] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260807 19:49:54.284258 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.285315 8902 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:54.285557 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.289604 8915 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:54.279908 12662 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:54.286850 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 57058.5us
I20260807 19:49:54.280653 12780 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:54.283104 12614 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:49:54.283377 12612 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:54.287408 12462 consensus_queue.cc:1059] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260807 19:49:54.288601 12759 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:54.288494 12784 log.cc:927] Time spent T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.081s user 0.001s sys 0.000s
I20260807 19:49:54.289182 12767 raft_consensus.cc:3055] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
W20260807 19:49:54.293092 12490 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.527s user 0.011s sys 0.000s
I20260807 19:49:54.289407 12687 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:54.289989 12775 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:54.312244 12767 raft_consensus.cc:740] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:54.316715 12767 consensus_queue.cc:260] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:54.317410 12699 consensus_meta.cc:287] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.520s user 0.011s sys 0.000s
I20260807 19:49:54.318321 12767 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:54.318622 12490 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.251829 12761 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.321990 12761 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:49:54.323002 12709 consensus_meta.cc:287] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.511s user 0.010s sys 0.000s
W20260807 19:49:54.327548 12738 consensus_meta.cc:287] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.553s user 0.002s sys 0.011s
W20260807 19:49:54.318257 12796 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
W20260807 19:49:54.312625 12790 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.131s user 0.001s sys 0.000s
I20260807 19:49:54.294296 12140 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:54.336741 12637 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:54.295503 12694 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.346616 12140 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.350873 12729 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.353482 12729 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:54.354984 12704 raft_consensus.cc:3055] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:54.355543 12704 raft_consensus.cc:740] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:54.361361 12704 consensus_queue.cc:260] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.364089 12704 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:54.370298 12681 consensus_queue.cc:260] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.371888 12681 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:54.382411 12490 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:54.386132 12808 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.062s user 0.001s sys 0.002s
W20260807 19:49:54.428373 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.434007 8780 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:54.435058 8775 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260807 19:49:54.435695 8775 pending_rounds.cc:85] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:54.436117 8775 pending_rounds.cc:107] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:54.436693 8775 raft_consensus.cc:2889] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:54.437256 8776 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
W20260807 19:49:54.454605 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.460575 8789 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.542923 8789 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:54.557960 12466 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:54.558911 12466 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.567521 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.568873 8906 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:54.573442 12140 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:54.574499 12140 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.576922 8968 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:54.384872 12807 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:54.581761 12807 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:54.581955 12811 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:54.583153 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:54.583050 12811 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.584864 8910 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260807 19:49:54.587225 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 309728us
I20260807 19:49:54.590554 12466 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:54.587419 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 354835us
I20260807 19:49:54.593509 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.595588 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:49:54.596529 12638 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:49:54.599025 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:54.602242 12438 raft_consensus.cc:2764] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260807 19:49:54.606747 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:54.607837 9892 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:49:54.608947 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.599326 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:54.615420 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.599246 12638 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:54.618881 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.618311 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.617529 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:54.633193 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.616078 12140 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:54.651198 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 70) took 1925 ms. Trace:
W20260807 19:49:54.651485 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 56169.9us
I20260807 19:49:54.658150 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.651819 8629 rpcz_store.cc:276] 0807 19:49:52.723536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:52.723702 (+ 166us) service_pool.cc:224] Handling call
0807 19:49:54.648648 (+1924946us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:53.904485 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:54.014571 (+110086us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:54.019416 (+ 4845us) write_op.cc:435] Acquired schema lock
0807 19:49:54.021857 (+ 2441us) tablet.cc:662] Decoding operations
0807 19:49:54.101734 (+ 79877us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:54.107958 (+ 6224us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:54.118265 (+ 10307us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:49:54.149341 (+ 31076us) tablet.cc:701] Row locks acquired
0807 19:49:54.156030 (+ 6689us) write_op.cc:260] PREPARE: finished
0807 19:49:54.189844 (+ 33814us) write_op.cc:270] Start()
0807 19:49:54.206918 (+ 17074us) write_op.cc:276] Timestamp: P: 1786132194176029 usec, L: 0
0807 19:49:54.210810 (+ 3892us) op_driver.cc:348] REPLICATION: starting
0807 19:49:54.257555 (+ 46745us) log.cc:844] Serialized 594 byte log entry
0807 19:49:54.360144 (+102589us) op_driver.cc:464] REPLICATION: finished
0807 19:49:54.531022 (+170878us) write_op.cc:301] APPLY: starting
0807 19:49:54.541810 (+ 10788us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:54.568201 (+ 26391us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:54.568427 (+ 226us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:54.568906 (+ 479us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:54.569329 (+ 423us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:49:54.569393 (+ 64us) write_op.cc:312] APPLY: finished
0807 19:49:54.569849 (+ 456us) log.cc:844] Serialized 22 byte log entry
0807 19:49:54.570210 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:54.570415 (+ 205us) write_op.cc:454] Released schema lock
0807 19:49:54.570483 (+ 68us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:54.571285 (+ 802us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:54.571330 (+ 45us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":608,"mutex_wait_us":3606,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":170337,"apply.run_cpu_time_us":8282,"apply.run_wall_time_us":40342,"num_ops":1,"prepare.queue_time_us":3150,"prepare.run_cpu_time_us":4705,"prepare.run_wall_time_us":374318,"replication_time_us":149112,"thread_start_us":169913,"threads_started":1}]]}
I20260807 19:49:54.651567 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.665289 8911 raft_consensus.cc:1275] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260807 19:49:54.668910 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.666013 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.669958 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.670856 8901 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:49:54.673498 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 132275us
I20260807 19:49:54.673795 12346 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:54.674851 8910 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:54.583729 12807 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.678864 12831 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:54.676209 8968 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:54.680708 12638 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:54.688385 12831 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:54.688769 12831 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:54.689987 12831 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.691216 12831 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:54.693924 12811 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:54.702834 12820 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:54.703352 12820 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:54.703683 12820 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:54.704845 8913 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:54.705986 8906 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:54.704131 12140 consensus_queue.cc:1059] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260807 19:49:54.713001 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:54.713620 12666 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:54.714057 12666 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:54.714629 8968 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:54.715723 12730 consensus_meta.cc:287] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.583s user 0.001s sys 0.011s
I20260807 19:49:54.715292 12666 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.681306 8740 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:54.719024 12666 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:54.719817 12438 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:54.686515 9892 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:54.729949 8915 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:54.734543 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.735808 8913 raft_consensus.cc:2393] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 5: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:54.736835 12612 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:49:54.737377 12612 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:54.737744 12612 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:49:54.744716 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.744617 8911 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:54.716228 12730 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:54.760247 8737 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:54.774740 9885 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 7. Ops: []
I20260807 19:49:54.772408 12609 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:54.778908 12140 consensus_queue.cc:1059] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260807 19:49:54.782428 12609 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:54.798605 12609 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:54.811231 8906 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:49:54.813259 8780 raft_consensus.cc:1240] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 2. Current term is 3. Ops: []
W20260807 19:49:54.819317 12614 consensus_meta.cc:287] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.504s user 0.005s sys 0.010s
I20260807 19:49:54.822100 12614 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.811305 12775 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:54.824638 12807 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:54.833348 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.838107 12638 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:54.838984 12638 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.833917 12775 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:54.857283 8968 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:54.857517 12614 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:54.864542 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:54.868803 12140 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:54.869527 12140 raft_consensus.cc:697] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:54.870772 12140 consensus_queue.cc:237] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.873813 12750 raft_consensus.cc:3055] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:54.874269 12750 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:54.875420 12750 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.876099 12834 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:54.876583 8783 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
W20260807 19:49:54.891379 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:54.867751 8628 catalog_manager.cc:5697] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
W20260807 19:49:54.904458 12761 consensus_meta.cc:287] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.507s user 0.011s sys 0.005s
I20260807 19:49:54.917306 12638 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:54.918502 12662 consensus_meta.cc:287] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.546s user 0.005s sys 0.010s
I20260807 19:49:54.926290 12750 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:49:54.942171 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.943516 8628 catalog_manager.cc:5697] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 6, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:54.940707 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 57826.2us
I20260807 19:49:54.955006 12666 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:54.956512 12775 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.954998 8902 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:54.935962 12662 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
W20260807 19:49:54.969327 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 92222.8us
I20260807 19:49:54.952322 8628 catalog_manager.cc:5697] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 5 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:54.971504 8628 catalog_manager.cc:5697] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:54.981775 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:54.982749 8901 raft_consensus.cc:2393] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 8: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:54.982107 8628 catalog_manager.cc:5697] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:54.984458 12820 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:54.985457 12813 consensus_queue.cc:1059] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:49:54.986486 12809 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 19:49:54.987833 12815 raft_consensus.cc:3055] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:54.987944 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.988260 12815 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:54.989383 12815 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.990734 12815 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:54.986692 12813 raft_consensus.cc:3055] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Stepping down as leader of term 2
I20260807 19:49:54.993162 12813 raft_consensus.cc:740] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:54.993741 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.994254 12813 consensus_queue.cc:260] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:54.995806 12813 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:54.996829 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:54.997980 12612 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.000453 9022 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:49:55.003309 12662 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260807 19:49:55.005139 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.006093 12609 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.010988 8901 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:54.986716 12820 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:55.015283 8628 catalog_manager.cc:5697] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 5 to 6, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:54.971261 12834 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:55.020529 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:49:55.016088 8903 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:49:55.022750 8901 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
W20260807 19:49:55.031771 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:55.025199 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 55881us
I20260807 19:49:55.000217 12612 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:55.032963 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:55.037595 8901 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:55.034143 12609 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:55.050704 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.051694 8775 raft_consensus.cc:2410] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:49:55.053292 12847 consensus_queue.cc:1059] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:49:55.054311 12847 raft_consensus.cc:3055] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:55.054853 12847 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.055812 12847 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.050545 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.054388 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.057120 8780 raft_consensus.cc:2393] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:55.057405 8783 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:55.058584 12847 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:55.059556 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:55.061244 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.063310 8775 raft_consensus.cc:2410] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260807 19:49:55.061275 8628 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 5 to 6, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:55.061643 8901 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:55.066454 8628 catalog_manager.cc:5697] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 6, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:55.073711 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.077615 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.082646 8628 catalog_manager.cc:5697] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:55.086714 8783 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:49:55.086867 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.092101 8972 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:55.097384 8780 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 7. Ops: []
I20260807 19:49:55.098620 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.148003 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:55.157852 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 69486us
I20260807 19:49:55.161890 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.163710 8775 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
W20260807 19:49:55.165845 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 48929us
I20260807 19:49:55.165305 8628 catalog_manager.cc:5697] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 5, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 5 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:49:55.168299 8972 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:55.169759 8789 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:55.172914 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.163426 12609 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:55.183851 8790 raft_consensus.cc:2393] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:55.175278 12346 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:49:55.176162 12775 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:55.188123 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.196481 8779 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:49:55.198864 12857 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.107s user 0.004s sys 0.000s
I20260807 19:49:55.201327 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.197522 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.202450 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.202744 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.203894 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:55.214547 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.196775 12775 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.215833 8972 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:55.201401 12490 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:55.221174 9885 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:55.222316 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.224949 8779 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:49:55.227314 8789 raft_consensus.cc:2368] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:55.233134 12556 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:55.232939 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.238775 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.239598 8779 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.240130 8787 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 9
I20260807 19:49:55.257088 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.262058 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.263139 8775 raft_consensus.cc:2410] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:49:55.236724 12490 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.268805 12775 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.269747 12490 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.270735 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:55.274892 8779 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:55.272322 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:55.286952 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 71286us
I20260807 19:49:55.287541 8968 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:55.288250 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.289520 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:55.296499 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:55.309581 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.318749 12490 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:55.319303 12490 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.319715 12490 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.320062 8788 raft_consensus.cc:1275] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260807 19:49:55.324663 8788 pending_rounds.cc:85] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:49:55.325191 8788 pending_rounds.cc:107] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260807 19:49:55.325560 8788 raft_consensus.cc:2889] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:55.327072 12638 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.327747 12638 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.330456 12638 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.293279 12834 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:55.331024 12834 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:55.331166 8787 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:49:55.331526 12834 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.332001 12490 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:55.335270 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.336067 8780 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
W20260807 19:49:55.336798 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 64566.9us
I20260807 19:49:55.337674 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.338649 12438 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:55.300277 12346 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:55.339069 12438 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.325815 8972 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.339805 12438 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:55.342764 12834 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.345460 12834 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:55.348856 12438 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.352444 12438 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.339005 8740 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:55.358950 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 62597.7us
I20260807 19:49:55.339721 12346 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:55.360548 12750 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:49:55.374020 8968 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:55.368018 12490 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.370954 12346 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:55.373488 12834 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:55.369413 12438 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:55.380532 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:55.382244 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 54922.4us
I20260807 19:49:55.383575 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:55.384003 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 86117.4us
I20260807 19:49:55.384729 9892 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260807 19:49:55.384966 8972 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.386276 12609 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:55.379506 12750 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.386756 12831 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 19:49:55.380486 12811 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:55.388021 12811 raft_consensus.cc:697] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.386714 12638 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:55.388654 12638 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.389043 12811 consensus_queue.cc:237] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.389640 12438 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.388864 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:55.391325 12556 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:49:55.391848 12556 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.390178 12638 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:55.386482 12775 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:55.393401 12775 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.393990 12775 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:55.393772 12556 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.386288 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:55.390075 12438 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:55.386422 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.397033 12807 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:49:55.429615 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:49:55.433146 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:49:55.446056 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 59746.4us
W20260807 19:49:55.450507 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.428835 12834 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260807 19:49:55.455614 12612 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:49:55.388108 12750 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.458052 12490 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:49:55.475550 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.489467 12831 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:55.490097 12831 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:55.490504 12831 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.501147 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.503407 8849 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.504518 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:55.505323 8901 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:55.506601 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:55.510769 8776 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:55.512454 8901 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260807 19:49:55.516649 8968 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:55.517400 12681 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:49:55.519322 8902 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:55.520077 12704 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:49:55.527683 12556 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:55.467940 12140 consensus_queue.cc:1048] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260807 19:49:55.537261 8788 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:49:55.561566 8775 raft_consensus.cc:1275] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:55.493475 12807 consensus_queue.cc:1059] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
W20260807 19:49:55.564301 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 61005.5us
I20260807 19:49:55.549688 12140 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260807 19:49:55.554991 12556 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.547209 12820 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260807 19:49:55.551446 12346 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.566375 12761 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:55.591066 9023 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260807 19:49:55.591523 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:55.602856 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:55.616549 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.618845 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.618821 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:55.617630 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.621999 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.620296 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.625128 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.585641 12839 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260807 19:49:55.632452 12346 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:55.587257 12638 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.591639 12538 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260807 19:49:55.617501 12609 consensus_queue.cc:1048] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260807 19:49:55.617743 12761 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.637665 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.640910 12438 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.641736 12807 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:55.649930 12811 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:55.656786 8789 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:55.643778 12846 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 19:49:55.671218 12869 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.130s user 0.001s sys 0.001s
I20260807 19:49:55.671804 12719 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260807 19:49:55.673859 12761 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.659366 12811 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.681905 12891 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260807 19:49:55.682222 12870 log.cc:927] Time spent T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.169s user 0.001s sys 0.001s
W20260807 19:49:55.682641 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 97308.5us
I20260807 19:49:55.683766 12638 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.684091 8849 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.684547 12638 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:55.685016 12638 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:55.685433 12638 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.686987 12893 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:55.688616 12893 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:55.685708 12881 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:55.685525 12761 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.689003 12893 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:55.697513 12831 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.699764 12831 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.681774 12750 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:55.691028 12775 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.688860 12811 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:55.691918 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 54367.5us
I20260807 19:49:55.690989 12881 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:55.693701 12438 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.700773 12750 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:49:55.706394 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 121361us
I20260807 19:49:55.712904 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.702342 12750 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.676245 12807 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.718640 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.719287 8903 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.701401 12811 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:55.718338 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.718080 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.720795 8901 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:55.721128 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.730218 8780 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:55.729020 8901 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:55.735795 8788 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:55.737565 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.739040 8776 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:49:55.739167 12831 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:55.739912 12831 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:55.740901 12831 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.744473 12839 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:49:55.716601 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:55.746311 9023 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:55.753544 8972 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:55.680576 12556 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.754881 8780 raft_consensus.cc:1240] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 3. Current term is 4. Ops: []
I20260807 19:49:55.756150 8911 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:55.757999 8856 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:55.755539 12681 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260807 19:49:55.762032 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:55.764894 8901 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:55.760610 12704 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:55.768608 8903 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
W20260807 19:49:55.769917 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 84304.4us
I20260807 19:49:55.712997 12881 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.764312 12820 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:55.749200 12891 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.790582 8911 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:55.791308 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.798388 8740 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.800117 8740 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:55.803809 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.806783 12895 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:55.808218 12775 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.808995 12775 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:55.809473 12775 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:55.810379 12775 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.769733 12704 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:55.812945 12704 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:55.815506 12893 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.792384 12891 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.799167 12761 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:55.825667 12891 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.827555 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.829123 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.830610 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.830996 12881 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:55.831336 12761 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:49:55.831756 12761 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:55.835148 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:55.834513 12813 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 3 to 7, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:55.836803 12662 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:55.841300 12750 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.843631 12750 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:55.832154 12761 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:49:55.849352 12638 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:55.855845 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:55.857542 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 52119.6us
I20260807 19:49:55.865778 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:55.862534 12807 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.868311 8629 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 5 to 8, leader changed from 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 8 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
W20260807 19:49:55.915530 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 52499.9us
W20260807 19:49:55.920146 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 89673.1us
I20260807 19:49:55.885850 12609 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:55.935006 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:55.922299 12891 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:55.927558 12490 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:55.923561 12834 consensus_queue.cc:1059] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260807 19:49:55.946177 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.939556 12895 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:55.929796 12847 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260807 19:49:55.941042 12609 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:55.960335 12897 consensus_queue.cc:1059] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260807 19:49:55.942871 12891 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.960754 12704 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.962332 12638 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.963053 12891 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.963186 12704 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.961104 12905 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:55.967073 12893 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:55.968894 12662 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:55.969514 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.970988 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:55.971486 12761 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.960080 12609 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:55.933136 12846 consensus_queue.cc:1059] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
W20260807 19:49:56.038925 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 68043.7us
I20260807 19:49:55.994349 12704 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.013211 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.993902 12891 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:56.030592 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 86171.9us
I20260807 19:49:56.033646 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:55.993127 12681 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:56.042303 8911 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:56.045831 8902 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:56.047554 8778 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:56.031709 12897 raft_consensus.cc:3055] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:49:56.039125 12893 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.061239 8788 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:49:56.041654 12662 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:56.071302 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:56.047528 12834 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.025s
I20260807 19:49:56.053102 12847 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:56.073887 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.053891 12897 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:49:56.083443 12917 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
I20260807 19:49:56.087873 9023 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:56.048988 12704 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.090407 8902 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:56.092778 8614 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 300f918f185249e6895ab57edd060d31 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:49:56.094482 12761 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.095824 12761 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.100517 12904 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260807 19:49:56.075289 12847 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.105130 8911 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:56.062721 12891 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.068442 12893 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.091058 12775 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.054054 12895 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:56.140699 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:49:56.124509 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 91505.3us
I20260807 19:49:56.142802 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:56.145637 8740 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:56.145885 12638 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:56.149093 12882 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:56.149739 12882 raft_consensus.cc:697] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:56.150837 12882 consensus_queue.cc:237] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.152756 12638 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:56.104434 12761 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.153874 12638 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.155726 12761 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:56.171521 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 84320.7us
I20260807 19:49:56.164903 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:56.185803 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 134392us
I20260807 19:49:56.169821 12662 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.187467 12846 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260807 19:49:56.201297 12662 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.203153 12891 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.202577 8911 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 3 to 7, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:56.143703 12775 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.201449 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:56.211302 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:56.211570 12931 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.212160 12931 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.214579 12931 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.214882 12893 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.216857 12905 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:56.203440 12662 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.223308 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.211292 12929 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:56.240638 12931 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.247167 12939 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.247097 12931 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.198122 12847 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.228869 12761 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.235942 12891 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.267742 8972 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:56.191303 12897 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.268265 12490 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.279999 9017 raft_consensus.cc:2955] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 3 to 7, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:49:56.281335 12820 raft_consensus.cc:3055] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:49:56.281881 12820 raft_consensus.cc:740] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.282833 12820 consensus_queue.cc:260] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.284168 12820 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:56.286571 12809 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.287185 12809 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.288424 12609 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:56.289353 12662 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:56.289971 12662 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.291100 12775 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.292155 12490 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:56.292271 12662 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:56.296908 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:56.268111 12847 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:56.255017 12931 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.301875 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.289963 12938 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.293844 12809 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.297333 12893 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.298780 12704 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:56.326763 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 139968us
I20260807 19:49:56.270210 12761 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:56.350852 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 83191.9us
I20260807 19:49:56.351900 8972 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:56.247779 12939 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.345610 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:56.329217 9850 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:56.309511 12897 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:56.300388 12891 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.364722 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.371539 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.347005 12612 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:56.377111 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.377727 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.378521 8788 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:49:56.370330 12704 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:49:56.370240 12809 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.387534 8856 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:56.345852 12893 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:56.389274 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 43787us
I20260807 19:49:56.328425 12938 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.388883 12931 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.407768 8849 leader_election.cc:400] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4.
I20260807 19:49:56.392156 12952 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:56.402200 12944 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.075s user 0.001s sys 0.000s
I20260807 19:49:56.398437 12612 raft_consensus.cc:697] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.401541 12929 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:56.421854 12939 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.412384 12919 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:56.405109 12609 raft_consensus.cc:2804] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:56.423808 8911 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260807 19:49:56.422086 12945 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260807 19:49:56.436239 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:56.441918 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.442488 12894 raft_consensus.cc:3055] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:49:56.396309 12809 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.450989 8911 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:49:56.453037 12761 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.459998 12490 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.448880 12894 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.469233 12894 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.470296 12894 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:56.471666 8789 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:56.473745 12490 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.474038 12609 raft_consensus.cc:697] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.475153 12609 consensus_queue.cc:237] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:56.480633 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 86886.5us
I20260807 19:49:56.425530 12929 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:56.488570 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:56.489144 8902 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:56.423420 12931 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.494078 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.497761 8902 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:56.495702 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.511509 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.485430 12929 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:56.517017 9018 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:49:56.435591 12952 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.529150 8776 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
W20260807 19:49:56.532871 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 54507.1us
I20260807 19:49:56.532382 8856 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:56.540900 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 43576.7us
W20260807 19:49:56.541529 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 189746us
I20260807 19:49:56.541690 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:56.544234 8902 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:56.547466 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:56.548213 8911 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:56.550439 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:56.556226 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:56.559145 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:56.559599 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:56.559854 9023 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:49:56.561748 8972 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:56.538870 12681 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.563201 12681 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.569509 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:56.554010 12939 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:56.581452 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:56.582185 9892 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:56.560029 12881 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:49:56.561429 12761 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:56.561748 12813 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.561808 12893 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.553769 12490 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260807 19:49:56.583511 8853 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:56.570370 12775 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:56.559191 12895 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260807 19:49:56.581393 12638 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:49:56.574241 12939 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.582993 12920 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:49:56.607280 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:56.596936 12704 consensus_queue.cc:1048] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.008s
I20260807 19:49:56.622486 12921 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:56.545236 12612 consensus_queue.cc:237] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.584714 12881 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:56.581959 12936 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:56.621141 12961 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.078s user 0.001s sys 0.000s
I20260807 19:49:56.622900 12905 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:49:56.605360 12938 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:49:56.632499 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 49069.5us
I20260807 19:49:56.601554 12893 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:56.621592 12775 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.625602 12939 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:56.597638 12813 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.650626 8856 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:56.656672 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:56.630255 12921 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:56.630703 12638 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:49:56.637898 12957 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.137s user 0.001s sys 0.000s
I20260807 19:49:56.690088 12950 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:56.690529 12950 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.690948 12950 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:56.692203 12952 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.694377 12346 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.695003 12346 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.697273 12346 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:56.633888 8972 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:56.698427 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 67466.1us
I20260807 19:49:56.699606 12704 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.655503 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.700974 12846 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:56.701535 12846 raft_consensus.cc:697] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.663151 12556 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.640102 12809 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:56.637295 12974 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.673688 12931 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.633111 12881 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:49:56.680970 12775 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.681842 12891 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.684010 12943 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260807 19:49:56.684756 12927 consensus_queue.cc:1059] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260807 19:49:56.637681 12681 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.689334 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:56.702252 12905 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.740595 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.740854 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.741024 8911 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:56.741281 8908 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:49:56.741727 12971 log.cc:927] Time spent T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.078s user 0.001s sys 0.000s
I20260807 19:49:56.629976 12920 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.700148 12704 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.742430 8911 raft_consensus.cc:1275] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:56.743623 8911 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:56.743881 12920 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.689047 12638 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.702088 12921 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:56.746031 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:56.744666 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:56.747473 8779 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:56.640429 12968 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:56.749960 12968 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.750969 12968 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.661935 8853 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:56.752739 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.721716 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.753487 9021 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:56.754292 12927 consensus_queue.cc:1243] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer 8e9ad2b5e2a94d4faff90af735c6d792 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260807 19:49:56.754866 12927 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:49:56.755184 12927 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.717530 12490 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.756021 12927 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.756647 12490 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.757416 12927 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:56.731135 12974 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.758948 12490 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.759560 12974 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.760900 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.741619 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.762727 12813 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:56.763547 12943 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:49:56.763923 12943 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:56.764901 12943 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:56.765756 8714 auto_rebalancer.cc:921] Could not move replica: Illegal state: not currently leader
W20260807 19:49:56.766191 8714 auto_rebalancer.cc:342] scheduled replica move failed to complete: Illegal state: not currently leader
I20260807 19:49:56.767647 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.659873 12893 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.773262 12893 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.774020 12905 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.766120 8908 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:56.776595 8913 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:49:56.732998 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:56.744166 12704 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.767560 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:56.789589 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:56.790344 8908 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:56.782152 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.795265 8901 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:49:56.740267 12953 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.216s user 0.000s sys 0.001s
I20260807 19:49:56.779389 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:56.788110 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 126278us
I20260807 19:49:56.779090 8910 raft_consensus.cc:2368] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8.
I20260807 19:49:56.731441 12966 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:56.766120 12943 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:56.651221 12936 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:56.801862 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 48839.5us
I20260807 19:49:56.825565 12538 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:49:56.826014 12966 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.829025 8915 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:56.805207 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:56.788450 12905 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.780627 12775 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:56.828382 12936 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.728484 12556 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.015123 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:56.893958 8856 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:57.021214 8908 raft_consensus.cc:2368] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6.
I20260807 19:49:56.929052 8913 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:49:56.971566 8915 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:56.896771 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:56.896880 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:57.029352 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 135471us
I20260807 19:49:56.906612 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:57.036177 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:57.044991 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:57.046576 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:56.821854 12761 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:49:56.905920 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:57.050251 12775 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.050648 12761 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:56.912314 12921 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.051410 12761 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:57.053128 12921 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.889743 9017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:56.853360 12638 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:57.049278 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:56.889196 12966 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:56.841724 12538 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:57.054380 9017 raft_consensus.cc:606] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:57.058866 9017 raft_consensus.cc:614] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:57.059195 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.054270 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.060107 8788 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:49:56.882771 12897 consensus_meta.cc:287] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.523s user 0.010s sys 0.000s
I20260807 19:49:57.060134 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.070825 8901 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:49:57.063067 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.069070 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 71) took 2235 ms. Trace:
W20260807 19:49:57.069591 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:57.069206 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:57.070364 9850 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:57.062608 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.072203 8901 raft_consensus.cc:1275] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:57.073256 8779 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:57.071954 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.075318 8901 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:49:57.073931 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.077557 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:57.261380 8789 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:49:57.072631 8629 rpcz_store.cc:276] 0807 19:49:54.819289 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:54.819448 (+ 159us) service_pool.cc:224] Handling call
0807 19:49:57.054137 (+2234689us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:55.972402 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:56.047361 (+ 74959us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:56.050117 (+ 2756us) write_op.cc:435] Acquired schema lock
0807 19:49:56.053460 (+ 3343us) tablet.cc:662] Decoding operations
0807 19:49:56.080332 (+ 26872us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:56.081815 (+ 1483us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:56.088705 (+ 6890us) tablet.cc:685] Acquiring locks for 1 operations
0807 19:49:56.128527 (+ 39822us) tablet.cc:701] Row locks acquired
0807 19:49:56.133140 (+ 4613us) write_op.cc:260] PREPARE: finished
0807 19:49:56.143192 (+ 10052us) write_op.cc:270] Start()
0807 19:49:56.166063 (+ 22871us) write_op.cc:276] Timestamp: P: 1786132196140106 usec, L: 0
0807 19:49:56.169300 (+ 3237us) op_driver.cc:348] REPLICATION: starting
0807 19:49:56.210648 (+ 41348us) log.cc:844] Serialized 532 byte log entry
0807 19:49:56.491011 (+280363us) op_driver.cc:464] REPLICATION: finished
0807 19:49:56.622740 (+131729us) write_op.cc:301] APPLY: starting
0807 19:49:56.622843 (+ 103us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:56.622945 (+ 102us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:56.654435 (+ 31490us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:56.731706 (+ 77271us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:56.769213 (+ 37507us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0807 19:49:56.769260 (+ 47us) write_op.cc:312] APPLY: finished
0807 19:49:56.769496 (+ 236us) log.cc:844] Serialized 22 byte log entry
0807 19:49:56.769726 (+ 230us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:56.769827 (+ 101us) write_op.cc:454] Released schema lock
0807 19:49:56.769872 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:56.770312 (+ 440us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:56.770340 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":91280,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":124951,"apply.run_cpu_time_us":2647,"apply.run_wall_time_us":147553,"num_ops":1,"prepare.queue_time_us":148,"prepare.run_cpu_time_us":5373,"prepare.run_wall_time_us":267568,"replication_time_us":277542,"thread_start_us":44686,"threads_started":1}]]}
I20260807 19:49:57.157264 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.069471 12538 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:57.172684 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.140189 12813 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:49:57.180445 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:56.929943 12939 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:57.140650 12944 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
I20260807 19:49:56.881057 12950 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.073751 12638 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:56.895802 12929 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.878393 12920 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:57.059337 12905 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:49:56.886371 12846 consensus_queue.cc:237] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:56.994563 12490 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.894632 12891 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.859854 12975 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:56.878134 12893 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:57.248179 9021 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:57.248896 8779 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:49:57.252028 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 141006us
I20260807 19:49:57.077806 9023 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:57.261523 8908 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 7: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:57.139744 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.543633 8783 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:57.454672 9008 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:57.329936 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.543498 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.543357 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.265372 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.547647 8628 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38434 (request call id 181) took 9663 ms. Trace:
I20260807 19:49:57.337029 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.549891 8787 raft_consensus.cc:2393] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:57.541335 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.552188 8790 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:57.555346 8778 raft_consensus.cc:2393] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 5: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:57.262375 9018 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:57.431756 8913 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:49:57.552558 9015 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:57.595842 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.296622 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.252621 12966 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.553210 12556 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:57.545384 12813 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:57.575645 12974 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.543756 12905 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:57.797739 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 420761us
W20260807 19:49:57.709781 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:57.797273 8913 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
W20260807 19:49:57.710227 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 127313us
I20260807 19:49:57.799085 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:57.622012 8906 raft_consensus.cc:2368] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:57.586411 12921 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.674446 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.337843 12895 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:57.709940 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:57.549068 8628 rpcz_store.cc:276] 0807 19:49:47.835471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:47.835830 (+ 359us) service_pool.cc:224] Handling call
0807 19:49:57.498658 (+9662828us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:55.519218 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:55.519858 (+ 640us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:55.519929 (+ 71us) write_op.cc:435] Acquired schema lock
0807 19:49:55.519946 (+ 17us) tablet.cc:662] Decoding operations
0807 19:49:55.521028 (+ 1082us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:55.521135 (+ 107us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:55.521162 (+ 27us) tablet.cc:685] Acquiring locks for 10 operations
0807 19:49:55.522057 (+ 895us) tablet.cc:701] Row locks acquired
0807 19:49:55.522080 (+ 23us) write_op.cc:260] PREPARE: finished
0807 19:49:55.522263 (+ 183us) write_op.cc:270] Start()
0807 19:49:55.522425 (+ 162us) write_op.cc:276] Timestamp: P: 1786132195522197 usec, L: 0
0807 19:49:55.522446 (+ 21us) op_driver.cc:348] REPLICATION: starting
0807 19:49:55.523211 (+ 765us) log.cc:844] Serialized 3999 byte log entry
0807 19:49:55.632610 (+109399us) op_driver.cc:464] REPLICATION: finished
0807 19:49:55.721579 (+ 88969us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f68784e0a5c 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687f05a8d0
0807 19:49:55.786634 (+ 65055us) write_op.cc:301] APPLY: starting
0807 19:49:55.786814 (+ 180us) tablet.cc:1367] starting BulkCheckPresence
0807 19:49:55.787219 (+ 405us) tablet.cc:1370] finished BulkCheckPresence
0807 19:49:55.787265 (+ 46us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:49:55.826802 (+ 39537us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:49:55.840382 (+ 13580us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0807 19:49:55.840451 (+ 69us) write_op.cc:312] APPLY: finished
0807 19:49:55.841850 (+ 1399us) log.cc:844] Serialized 76 byte log entry
0807 19:49:55.842756 (+ 906us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:55.842900 (+ 144us) write_op.cc:454] Released schema lock
0807 19:49:55.842956 (+ 56us) write_op.cc:341] FINISH: Updating metrics
0807 19:49:55.844569 (+ 1613us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:49:55.844602 (+ 33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":264512,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":153574,"apply.run_cpu_time_us":4021,"apply.run_wall_time_us":57988,"num_ops":10,"prepare.queue_time_us":140237,"prepare.run_cpu_time_us":4330,"prepare.run_wall_time_us":4329,"replication_time_us":109980,"spinlock_wait_cycles":7482112,"thread_start_us":147402,"threads_started":2}]]}
I20260807 19:49:57.872007 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:57.879820 9019 raft_consensus.cc:2410] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260807 19:49:57.760634 12761 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.700s user 0.007s sys 0.007s
W20260807 19:49:57.757846 12985 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.420s user 0.000s sys 0.001s
I20260807 19:49:57.573526 12490 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.858345 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:57.901479 12965 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.901120 8630 catalog_manager.cc:5697] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 4 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 5 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:57.903436 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:57.726022 12897 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.577632 12891 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.317540 12881 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.750777 12927 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.577260 12975 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.708285 12929 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:57.914726 8775 raft_consensus.cc:3060] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:49:57.911850 9022 consensus_meta.cc:287] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.666s user 0.006s sys 0.005s
I20260807 19:49:57.849409 12813 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:57.946026 8913 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:57.950407 9017 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260807 19:49:57.861526 12905 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:57.982120 9022 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:57.989171 8630 catalog_manager.cc:5697] T e10d78176f8545489a4700b7bd08fec3 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 8, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 8 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:57.921459 12984 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.247s user 0.001s sys 0.000s
I20260807 19:49:57.865128 12974 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:57.823897 12966 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.008090 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.027954 8913 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:49:58.029394 8913 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:57.895224 12921 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.036192 8779 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:49:58.015067 12968 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:57.593034 12920 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.048662 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.030310 13011 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.048120 12950 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:57.974540 12965 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:58.048512 8854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s user 0.018s sys 0.025s
W20260807 19:49:58.074740 8854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s user 0.019s sys 0.026s
I20260807 19:49:58.057606 8630 catalog_manager.cc:5697] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 6, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:58.061058 8790 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:58.065482 9023 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:49:57.986410 12897 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:57.604511 12893 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.009088 12927 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.074782 13011 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.084549 13010 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.065766 12638 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:58.065675 12939 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.092999 8630 catalog_manager.cc:5697] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:58.094753 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 339165us
I20260807 19:49:58.092695 12975 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:58.095980 12994 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.092s user 0.000s sys 0.001s
I20260807 19:49:58.095624 13010 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.085209 13005 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:57.920214 12761 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:49:58.099885 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 241587us
W20260807 19:49:58.101261 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 342152us
I20260807 19:49:57.911666 12895 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.113879 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.054373 12968 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.117781 8968 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:58.103031 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.121984 9023 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:58.118618 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:58.119939 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:58.124261 8913 raft_consensus.cc:2393] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:58.126890 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.127947 9008 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:58.131772 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:58.133268 8630 catalog_manager.cc:5697] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 7, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.136234 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.137368 9020 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:58.137830 8913 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:58.107568 12681 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:49:58.156219 8788 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.859s user 0.011s sys 0.031s
W20260807 19:49:58.156723 8788 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.859s user 0.011s sys 0.031s
I20260807 19:49:58.119778 12882 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.145290 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.145629 8630 catalog_manager.cc:5697] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 5, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:58.185590 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 292618us
I20260807 19:49:58.222684 8630 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 7, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.171651 9020 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260807 19:49:58.250916 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 321367us
W20260807 19:49:58.158867 12538 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.573s user 0.008s sys 0.002s
I20260807 19:49:58.251107 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.254936 9021 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:49:58.129494 13005 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.163133 12681 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.324375 9018 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
W20260807 19:49:58.426059 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 308348us
I20260807 19:49:58.421301 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:58.275075 9021 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:58.433272 9008 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:49:58.428920 12974 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:58.453193 12974 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.453851 12974 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.307860 8630 catalog_manager.cc:5697] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 4, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.457315 8630 catalog_manager.cc:5697] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 7, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.439128 12891 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.440598 12966 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.276856 9017 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:58.443789 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.464663 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.466053 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.466872 9020 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:58.446916 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:58.468740 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.437119 12775 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.309230 13025 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:58.448869 12490 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.444140 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1186) took 1739 ms. Trace:
I20260807 19:49:58.444411 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.445662 12895 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:58.472863 9008 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
W20260807 19:49:58.312476 13022 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.152s user 0.001s sys 0.000s
I20260807 19:49:58.475741 8908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1569) took 1087 ms. Trace:
W20260807 19:49:58.397445 12990 log.cc:927] Time spent T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.131s user 0.000s sys 0.001s
I20260807 19:49:58.474108 8903 rpcz_store.cc:276] 0807 19:49:56.704376 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:56.704499 (+ 123us) service_pool.cc:224] Handling call
0807 19:49:56.750298 (+ 45799us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:56.777095 (+ 26797us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:56.777128 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:56.777620 (+ 492us) log.cc:844] Serialized 26 byte log entry
0807 19:49:57.223610 (+445990us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:57.223658 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:57.223743 (+ 85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:58.394034 (+1170291us) raft_consensus.cc:1685] finished
0807 19:49:58.394057 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.443937 (+ 49880us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53364,"raft.queue_time_us":37276,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":19832,"thread_start_us":445535,"threads_started":1,"wal-append.queue_time_us":1384712}
W20260807 19:49:58.314211 12994 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260807 19:49:58.461392 8630 catalog_manager.cc:5697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.490302 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.497192 9008 raft_consensus.cc:2368] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:49:58.498203 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.499378 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.499926 9008 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:58.503243 12891 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.503973 12891 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.510763 12891 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.517135 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:58.546171 8903 raft_consensus.cc:2368] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:49:58.520867 13011 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.456144 12974 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.309710 13023 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:58.277662 12538 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.534315 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1187) took 1817 ms. Trace:
I20260807 19:49:58.534925 8904 raft_consensus.cc:1217] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[5.4-5.4] Dedup: 5.4->[]
I20260807 19:49:58.470468 13025 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:58.542366 9019 consensus_meta.cc:287] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.525s user 0.007s sys 0.005s
W20260807 19:49:58.543097 8910 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.186s user 0.023s sys 0.017s
I20260807 19:49:58.562237 9008 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
W20260807 19:49:58.488257 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 3def68aebfd04eeab681e41691bae18c (from a38f4d6088cf4c8cb510200ca0555801 to 8e9ad2b5e2a94d4faff90af735c6d792) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:49:58.247908 12882 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.265069 13016 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:58.568881 8630 catalog_manager.cc:5697] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:58.581069 8910 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.265s user 0.024s sys 0.017s
I20260807 19:49:58.495965 13010 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.568959 9019 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:49:58.571722 8904 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:49:58.572642 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:58.592769 8903 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:58.595737 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.597193 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.598003 9019 raft_consensus.cc:2410] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 19:49:58.598791 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.599352 9019 raft_consensus.cc:2376] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8. Re-sending same reply.
W20260807 19:49:58.562489 12905 consensus_meta.cc:287] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.546s user 0.002s sys 0.009s
I20260807 19:49:58.575264 13025 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.575824 13030 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:58.576862 8906 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1547) took 1855 ms (client timeout 1511 ms). Trace:
I20260807 19:49:58.579993 13041 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.580791 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.582762 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1559) took 1565 ms. Trace:
W20260807 19:49:58.654239 8906 rpcz_store.cc:269] 0807 19:49:56.721029 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:56.723416 (+ 2387us) service_pool.cc:224] Handling call
0807 19:49:58.576543 (+1853127us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b540035f2d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:58.576722 (+ 179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1963009920}
I20260807 19:49:58.557828 12846 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 19:49:58.600301 8915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s user 0.016s sys 0.012s
I20260807 19:49:58.624143 13010 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.566752 8902 rpcz_store.cc:276] 0807 19:49:56.717405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:56.728437 (+ 11032us) service_pool.cc:224] Handling call
0807 19:49:56.740099 (+ 11662us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:49:56.820625 (+ 80526us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:56.820656 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:49:56.828129 (+ 7473us) log.cc:844] Serialized 52 byte log entry
0807 19:49:57.058835 (+230706us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:57.305379 (+246544us) log.cc:844] Serialized 14 byte log entry
0807 19:49:57.384974 (+ 79595us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:57.385079 (+ 105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:58.534040 (+1148961us) raft_consensus.cc:1685] finished
0807 19:49:58.534064 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.534239 (+ 175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":98879,"prepare.queue_time_us":839785,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":343000,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":13000,"spinlock_wait_cycles":24830848,"thread_start_us":425425,"threads_started":3,"wal-append.queue_time_us":1297701}
W20260807 19:49:58.657270 8915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s user 0.017s sys 0.012s
I20260807 19:49:58.586704 12775 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:58.581641 12813 consensus_meta.cc:287] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.564s user 0.011s sys 0.001s
I20260807 19:49:58.600214 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:58.601316 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 345957us
I20260807 19:49:58.624751 8630 catalog_manager.cc:5697] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 7, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 7 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:49:58.655201 8911 rpcz_store.cc:276] 0807 19:49:57.017543 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.017686 (+ 143us) service_pool.cc:224] Handling call
0807 19:49:57.018007 (+ 321us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:57.018390 (+ 383us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:57.018418 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:57.018999 (+ 581us) log.cc:844] Serialized 26 byte log entry
0807 19:49:57.076399 (+ 57400us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:57.382838 (+306439us) log.cc:844] Serialized 14 byte log entry
0807 19:49:57.996206 (+613368us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:57.996285 (+ 79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:58.582542 (+586257us) raft_consensus.cc:1685] finished
0807 19:49:58.582563 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.582697 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173226,"prepare.queue_time_us":822521,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":240459,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":56334,"thread_start_us":595131,"threads_started":3,"wal-append.queue_time_us":835127}
I20260807 19:49:58.666581 9019 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:58.666935 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:58.663563 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:58.669188 8902 raft_consensus.cc:1275] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:58.667721 8911 raft_consensus.cc:2368] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:58.630743 12905 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.671577 8630 catalog_manager.cc:5697] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.681129 8903 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:58.591151 13016 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.685060 8630 catalog_manager.cc:5697] T f02f58b5ca51497592868892f144492a P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.660884 12846 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:58.693141 8630 catalog_manager.cc:5697] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.694262 12846 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:49:58.696823 13045 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.697480 13045 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.699874 13045 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.701013 8630 catalog_manager.cc:5697] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.706761 12893 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.703576 13005 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.662959 13030 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.707924 12921 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:58.708752 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.709482 12538 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.710263 9023 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:58.712034 9023 raft_consensus.cc:1275] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:58.712249 8915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1565) took 1655 ms. Trace:
I20260807 19:49:58.710263 9017 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260807 19:49:58.713299 9017 pending_rounds.cc:85] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Aborting all ops after (but not including) 4
I20260807 19:49:58.713570 9017 pending_rounds.cc:107] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260807 19:49:58.713858 9017 raft_consensus.cc:2889] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:58.712671 8915 rpcz_store.cc:276] 0807 19:49:57.057168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.372723 (+315555us) service_pool.cc:224] Handling call
0807 19:49:57.373124 (+ 401us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:57.865496 (+492372us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:57.865532 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:57.908250 (+ 42718us) log.cc:844] Serialized 26 byte log entry
0807 19:49:58.657776 (+749526us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:58.677316 (+ 19540us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.677597 (+ 281us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:58.677679 (+ 82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:58.712046 (+ 34367us) raft_consensus.cc:1685] finished
0807 19:49:58.712067 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.712188 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3273,"prepare.queue_time_us":24936,"raft.queue_time_us":229567,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":13078,"spinlock_wait_cycles":38814848,"thread_start_us":1114466,"threads_started":2,"wal-append.queue_time_us":715953}
I20260807 19:49:58.665416 13010 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.716326 12927 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.668677 12775 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.716859 12897 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.718123 12761 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:58.687119 12968 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.719489 13023 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:58.664907 13041 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.720747 9020 raft_consensus.cc:2368] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:49:58.721292 9019 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:49:58.720180 13023 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.720698 12775 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.676854 12813 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:58.712671 12882 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.722959 9892 raft_consensus.cc:2393] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:58.724762 13041 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.726207 8630 catalog_manager.cc:5697] T ea5ba8c8385a404799a0be85834f98f4 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 4 to 7, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:58.721586 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:58.728797 8630 catalog_manager.cc:5697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 7, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:49:58.724776 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:49:58.724880 13023 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:58.705296 12846 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.726217 13010 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.734453 9008 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:49:58.735312 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.744684 12936 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.745445 13005 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.746274 12761 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.747238 13038 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.791632 8630 catalog_manager.cc:5697] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 5, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:49:58.761695 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 326659us
I20260807 19:49:58.806085 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.778386 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53512 (request call id 174) took 1346 ms. Trace:
W20260807 19:49:58.787742 8740 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.007s sys 0.017s
I20260807 19:49:58.766960 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1917) took 1481 ms. Trace:
I20260807 19:49:58.751575 8856 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:58.808166 8740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s user 0.007s sys 0.017s
W20260807 19:49:58.808629 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 57126.7us
I20260807 19:49:58.771306 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:58.789810 8778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.007s sys 0.014s
W20260807 19:49:58.810355 8778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s user 0.014s sys 0.021s
I20260807 19:49:58.809253 8790 rpcz_store.cc:276] 0807 19:49:57.431863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.595326 (+163463us) service_pool.cc:224] Handling call
0807 19:49:58.778083 (+1182757us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b54000898d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:58.778244 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463429376}
I20260807 19:49:58.710265 13030 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.815783 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.747458 12882 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:58.817763 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 151754us
I20260807 19:49:58.853847 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1422) took 1601 ms. Trace:
W20260807 19:49:58.994240 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 186933us
I20260807 19:49:59.062523 9017 raft_consensus.cc:2410] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260807 19:49:58.854197 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1563) took 1797 ms. Trace:
W20260807 19:49:58.845321 8740 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.195:35801 timed out after 1.868s (SENT) [suppressed 5 similar messages]
W20260807 19:49:59.189841 8776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1419) took 1938 ms (client timeout 1890 ms). Trace:
I20260807 19:49:58.850225 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:59.184288 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 121586us
I20260807 19:49:59.184999 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 1918) took 1936 ms. Trace:
W20260807 19:49:58.845412 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 397558us
I20260807 19:49:58.482720 8908 rpcz_store.cc:276] 0807 19:49:57.388502 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.706226 (+317724us) service_pool.cc:224] Handling call
0807 19:49:57.791506 (+ 85280us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:57.791764 (+ 258us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:58.135178 (+343414us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.475367 (+340189us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:58.475424 (+ 57us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:58.475491 (+ 67us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:58.475559 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.475681 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85438,"prepare.queue_time_us":107683,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":79177,"thread_start_us":597108,"threads_started":2}
I20260807 19:49:58.880245 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1558) took 1863 ms. Trace:
I20260807 19:49:59.201279 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.781172 12897 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.886785 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:58.991034 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:59.200098 8783 rpcz_store.cc:276] 0807 19:49:57.248492 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.248603 (+ 111us) service_pool.cc:224] Handling call
0807 19:49:57.543423 (+294820us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:57.552408 (+ 8985us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:59.184747 (+1632339us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54000965d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:59.184923 (+ 176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677330816}
W20260807 19:49:59.202849 8776 rpcz_store.cc:269] 0807 19:49:57.251244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:59.189571 (+1938327us) service_pool.cc:211] Skipping call since client already timed out
0807 19:49:59.189728 (+ 157us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260807 19:49:58.810400 8788 rpcz_store.cc:276] 0807 19:49:57.247477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.247589 (+ 112us) service_pool.cc:224] Handling call
0807 19:49:57.247853 (+ 264us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:57.248050 (+ 197us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:58.192673 (+944623us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.688124 (+495451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:58.688189 (+ 65us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:49:58.688257 (+ 68us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:58.688331 (+ 74us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.729261 (+ 40930us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54501,"prepare.queue_time_us":293575,"prepare.run_cpu_time_us":249,"prepare.run_wall_time_us":62337,"raft.queue_time_us":1150,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":26815,"spinlock_wait_cycles":2816,"thread_start_us":1332578,"threads_started":2}
W20260807 19:49:59.147203 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 85420.8us
I20260807 19:49:59.221768 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:59.222456 8913 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:49:59.064496 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:49:59.202327 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 357040us
I20260807 19:49:58.884639 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:58.862999 12813 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.861215 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:49:59.066346 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 277048us
I20260807 19:49:59.241641 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.201682 8904 rpcz_store.cc:276] 0807 19:49:57.016663 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.016790 (+ 127us) service_pool.cc:224] Handling call
0807 19:49:58.534802 (+1518012us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:58.631378 (+ 96576us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:58.880040 (+248662us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540020d350. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:58.880177 (+ 137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3490277248}
I20260807 19:49:58.860555 9885 raft_consensus.cc:2368] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6.
I20260807 19:49:59.063306 8901 rpcz_store.cc:276] 0807 19:49:57.056638 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.139423 (+ 82785us) service_pool.cc:224] Handling call
0807 19:49:57.261363 (+121940us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:57.722872 (+461509us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:49:58.125952 (+403080us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.652601 (+526649us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:58.662541 (+ 9940us) log.cc:844] Serialized 26 byte log entry
0807 19:49:58.662842 (+ 301us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:58.686451 (+ 23609us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.751856 (+ 65405us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:58.751931 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:58.853972 (+102041us) raft_consensus.cc:1685] finished
0807 19:49:58.854000 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.854133 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89813,"prepare.queue_time_us":199615,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":321690,"raft.run_cpu_time_us":512,"raft.run_wall_time_us":28349,"thread_start_us":1179492,"threads_started":3,"wal-append.queue_time_us":604208}
I20260807 19:49:59.245193 8972 leader_election.cc:400] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8.
I20260807 19:49:58.998875 12891 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:58.995921 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:59.215582 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.253650 8787 raft_consensus.cc:2393] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 7: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:59.254179 8787 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1420) took 2002 ms (client timeout 1835 ms). Trace:
W20260807 19:49:59.254622 8787 rpcz_store.cc:269] 0807 19:49:57.251505 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:58.859802 (+1608297us) service_pool.cc:224] Handling call
0807 19:49:59.254077 (+394275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:58.817531 13010 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:49:59.255445 8901 raft_consensus.cc:1275] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:49:59.254236 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:59.232048 13059 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:58.855607 13054 consensus_queue.cc:1048] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.394s
I20260807 19:49:59.257669 12974 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:59.257900 12638 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:59.258489 13047 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.258679 12983 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.259073 13047 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.259231 12983 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.261727 13047 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.227244 12897 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.255230 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.264767 12897 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:59.265197 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.225052 12975 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:59.266430 12975 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.266762 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:49:58.859172 9022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1141) took 1966 ms (client timeout 1868 ms). Trace:
W20260807 19:49:59.267961 9022 rpcz_store.cc:269] 0807 19:49:56.893154 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:56.905558 (+ 12404us) service_pool.cc:224] Handling call
0807 19:49:58.858895 (+1953337us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b540043c6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:58.859032 (+ 137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39348096}
I20260807 19:49:59.269652 8913 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:49:58.845481 13025 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.862740 12905 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.145219 12846 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:58.845252 12893 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:59.237617 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:59.238415 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.239470 8906 raft_consensus.cc:1240] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 1. Current term is 2. Ops: []
I20260807 19:49:58.847280 12999 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.163622 12882 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.242393 8908 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
W20260807 19:49:59.065174 8779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53512 (request call id 175) took 1562 ms (client timeout 1608 ms). Trace:
I20260807 19:49:59.168987 13007 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.163174 13016 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.247934 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:58.816262 12761 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.270397 12638 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.256935 13059 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:58.812703 12936 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.061749 8778 rpcz_store.cc:276] 0807 19:49:57.252000 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.601879 (+349879us) service_pool.cc:224] Handling call
0807 19:49:57.620772 (+ 18893us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:57.621204 (+ 432us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 19:49:57.751218 (+130014us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.035370 (+284152us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b18003b6200. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a342a 00007f687b60ee0a
0807 19:49:58.811310 (+775940us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.813252 (+ 1942us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:58.814184 (+ 932us) log.cc:844] Serialized 26 byte log entry
0807 19:49:58.814491 (+ 307us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:49:58.814756 (+ 265us) log.cc:844] Serialized 14 byte log entry
0807 19:49:58.847585 (+ 32829us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:58.847762 (+ 177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:58.853606 (+ 5844us) raft_consensus.cc:1685] finished
0807 19:49:58.853626 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:58.853775 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32539,"prepare.queue_time_us":182580,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":144,"raft.queue_time_us":578683,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":6941,"spinlock_wait_cycles":24465280,"thread_start_us":775367,"threads_started":2,"wal-append.queue_time_us":892320}
I20260807 19:49:58.748641 12775 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.947028 12968 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:59.290020 8972 leader_election.cc:400] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6.
I20260807 19:49:59.216548 9890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42580 (request call id 1) took 2150 ms. Trace:
I20260807 19:49:59.291921 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:49:59.373067 9892 raft_consensus.cc:2393] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:59.280020 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
W20260807 19:49:59.374346 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 94415.7us
I20260807 19:49:59.291926 8911 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:49:59.380622 13045 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:49:59.381286 13045 raft_consensus.cc:697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:59.382284 13045 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.229384 12991 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.290881 9015 raft_consensus.cc:2393] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:58.847610 12921 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:58.817194 13038 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.258287 12974 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.322896 12975 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.324309 12983 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:58.815991 13005 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.255071 8904 raft_consensus.cc:1275] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260807 19:49:59.228319 12881 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:49:59.325989 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:49:59.319492 9027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1150) took 1492 ms. Trace:
W20260807 19:49:59.319095 8779 rpcz_store.cc:269] 0807 19:49:57.502564 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.861546 (+358982us) service_pool.cc:224] Handling call
0807 19:49:59.064969 (+1203423us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540008b6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:59.065101 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366769024}
I20260807 19:49:59.329591 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1555) took 2565 ms. Trace:
W20260807 19:49:59.336647 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 46769.2us
I20260807 19:49:59.395411 8911 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:49:59.396284 9015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57870 (request call id 1577) took 1276 ms. Trace:
I20260807 19:49:59.329111 13004 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 19:49:59.266458 13058 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.415047 9015 rpcz_store.cc:276] 0807 19:49:58.120117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:58.946257 (+826140us) service_pool.cc:224] Handling call
0807 19:49:59.396210 (+449953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:49:59.413502 9027 rpcz_store.cc:276] 0807 19:49:57.826716 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:59.229152 (+1402436us) service_pool.cc:224] Handling call
0807 19:49:59.319413 (+ 90261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:49:59.405395 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 138729us
I20260807 19:49:59.415740 9850 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:49:59.293373 12891 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.419659 8972 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:49:59.405860 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 205936us
W20260807 19:49:59.426904 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 191839us
I20260807 19:49:59.426936 8849 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.328619 12893 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.427590 9017 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:49:59.435307 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.436196 9018 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
W20260807 19:49:59.434825 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 155396us
I20260807 19:49:59.430325 8740 leader_election.cc:400] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:49:59.438156 8740 leader_election.cc:403] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:59.438186 8853 leader_election.cc:400] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Vote denied by peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143) with higher term. Message: Invalid argument: T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6.
I20260807 19:49:59.397504 12809 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:59.331122 13007 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.396090 13010 raft_consensus.cc:697] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:59.328260 13025 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.386915 12775 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.328770 12999 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.389226 12968 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.478523 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:49:59.414556 8910 rpcz_store.cc:276] 0807 19:49:56.732568 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.053818 (+321250us) service_pool.cc:224] Handling call
0807 19:49:57.249492 (+195674us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:49:58.586765 (+1337273us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:58.586800 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:49:58.595293 (+ 8493us) log.cc:844] Serialized 26 byte log entry
0807 19:49:58.896129 (+300836us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:59.013468 (+117339us) log.cc:844] Serialized 14 byte log entry
0807 19:49:59.013748 (+ 280us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:59.013836 (+ 88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:49:59.183381 (+169545us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b48000c03d8. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6880560211 00007f68805572a7 00007f688055713a 00007f688064e07f 00007f688064dd47 00007f688066236c 00007f6880662318 00007f6880662298 00007f6880662253 00007f68806614fc 00007f687f589176
0807 19:49:59.273125 (+ 89744us) raft_consensus.cc:1685] finished
0807 19:49:59.295903 (+ 22778us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:49:59.296086 (+ 183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124380,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":288976,"raft.run_cpu_time_us":14335,"raft.run_wall_time_us":1609542,"spinlock_wait_cycles":42777984,"thread_start_us":1483555,"threads_started":2,"wal-append.queue_time_us":580839}
I20260807 19:49:59.608456 8853 leader_election.cc:403] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260807 19:49:59.601717 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 130214us
I20260807 19:49:59.404697 12921 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:49:59.610240 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 183382us
I20260807 19:49:59.412144 12965 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:59.415529 12983 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:59.624588 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 192593us
I20260807 19:49:59.630002 13063 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:49:59.626746 8849 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.628271 8789 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
W20260807 19:49:59.628644 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 150236us
I20260807 19:49:59.631896 12983 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.634179 12983 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.634699 12983 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:59.635164 12983 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.629805 12921 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.638288 12921 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 8
I20260807 19:49:59.640471 8972 leader_election.cc:400] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:59.608724 12846 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:59.654726 8972 leader_election.cc:403] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 19:49:59.435412 13012 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:49:59.405934 12991 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.461153 12905 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:59.647863 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.654266 12975 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143) with higher term. Message: Invalid argument: T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6.
W20260807 19:49:59.655221 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 230616us
I20260807 19:49:59.430135 13023 consensus_queue.cc:1048] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.227s
I20260807 19:49:59.655944 9021 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:49:59.635874 13065 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260807 19:49:59.372334 9890 rpcz_store.cc:276] 0807 19:49:57.053499 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:57.053679 (+ 180us) service_pool.cc:224] Handling call
0807 19:49:59.057101 (+2003422us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b5400750e20. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b581891 00007f687b609c43 00007f687b602dda 00007f68805fc2d9 00007f6879b46db4 00007f6879b46d42 00007f6879b46c71 00007f6879b46bec 00007f6879b45ea2 00007f687993e07c
0807 19:49:59.169739 (+112638us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113191296}
W20260807 19:49:59.667445 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 237787us
I20260807 19:49:59.624193 12809 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.437055 13040 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.225s
I20260807 19:49:59.463774 12881 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.689441 12881 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.690227 12891 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.691416 12809 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.675732 8915 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:49:59.691985 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.693645 12893 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.694445 12882 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.618930 12897 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260807 19:49:59.677086 8849 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.666160 12965 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5.
I20260807 19:49:59.670833 12846 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:49:59.687574 13038 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.691586 12881 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.706101 12968 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:49:59.702196 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 72797.8us
I20260807 19:49:59.684903 12975 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:59.709923 12846 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.711684 12968 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:49:59.677587 12905 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:59.714675 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.714759 12882 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:59.715425 8779 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:49:59.695526 13005 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.688376 13016 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:59.673897 12983 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.716521 13016 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.717052 8779 raft_consensus.cc:1240] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:49:59.718525 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.719215 8779 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:49:59.720631 12761 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.721802 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.693660 12991 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.720678 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:59.722963 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 82700.7us
I20260807 19:49:59.723254 8779 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:49:59.727458 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 50458.5us
I20260807 19:49:59.728278 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.728489 8783 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:59.729365 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:49:59.729972 8788 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:49:59.739972 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.432148 13058 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.672557 13012 raft_consensus.cc:697] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [term 4 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:49:59.744763 8776 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:49:59.740757 8778 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:59.680050 13069 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.722293 12974 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.758121 13059 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:59.769968 13016 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.770787 12936 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.772194 12999 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.745479 13038 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
W20260807 19:49:59.773695 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 45511.2us
I20260807 19:49:59.774490 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.776542 13005 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:59.777292 13005 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:49:59.729305 13075 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:49:59.778395 13075 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.761549 13069 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.782835 13069 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.785691 12638 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.790439 13038 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:49:59.791560 12681 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.776158 8849 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.795286 13075 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.716482 13007 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.797734 8788 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:49:59.788020 12920 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.801642 8783 raft_consensus.cc:1275] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260807 19:49:59.801434 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 99366.3us
I20260807 19:49:59.807117 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.807826 8778 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:59.808195 13025 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:49:59.810295 13058 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.811460 12775 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.752048 13010 consensus_queue.cc:237] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.816895 8740 leader_election.cc:400] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:59.817375 8740 leader_election.cc:403] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:49:59.818394 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:49:59.819464 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:49:59.825664 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 49572.4us
I20260807 19:49:59.826869 8849 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.837977 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:49:59.848362 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.851485 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:59.852231 8789 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1423) took 2594 ms (client timeout 1888 ms). Trace:
I20260807 19:49:59.852157 8904 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:59.858748 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:49:59.854800 8789 rpcz_store.cc:269] 0807 19:49:57.258053 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:58.886451 (+1628398us) service_pool.cc:224] Handling call
0807 19:49:59.852027 (+965576us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540008cad4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:49:59.852145 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943241472}
I20260807 19:49:59.861102 8778 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260807 19:49:59.861430 8778 pending_rounds.cc:85] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 3
I20260807 19:49:59.861621 8778 pending_rounds.cc:107] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:49:59.861807 8778 raft_consensus.cc:2889] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:49:59.859313 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.816668 13012 consensus_queue.cc:237] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:49:59.875335 8908 raft_consensus.cc:2393] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:59.844511 12775 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 5
I20260807 19:49:59.876523 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.877050 8913 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:49:59.877355 8902 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:49:59.878501 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:49:59.878975 8906 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:49:59.882277 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.885040 8910 raft_consensus.cc:2393] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 4: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:59.886451 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:49:59.887061 8783 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:49:59.887261 8906 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:49:59.885824 8901 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:59.832857 12681 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:49:59.844295 13058 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:49:59.894492 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.895951 8901 raft_consensus.cc:2393] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 5: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:49:59.851148 12950 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:49:59.859469 13069 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:49:59.859359 13025 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:49:59.901913 9018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:49:59.902498 9018 raft_consensus.cc:606] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:49:59.902832 9018 raft_consensus.cc:614] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:49:59.908205 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.908715 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.899797 13025 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:49:59.908259 12968 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:49:59.909376 13025 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:49:59.909721 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.909490 12968 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.910341 8903 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:59.911222 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.911767 8903 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:49:59.911777 12968 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:49:59.891183 12991 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:59.913872 8906 raft_consensus.cc:2393] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:49:59.913141 8904 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:59.919397 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.915681 8968 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:49:59.918915 8740 connection.cc:256] RPC call timeout handler was delayed by -2.0458270059898496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
I20260807 19:49:59.927500 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.927798 13093 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:49:59.927548 8740 connection.cc:256] RPC call timeout handler was delayed by -2.0067058440763503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18030000000000002s for a response
I20260807 19:49:59.928814 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.929107 13093 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.905349 13038 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.932686 13093 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:49:59.930004 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.932650 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.901136 13058 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:49:59.940768 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:49:59.945000 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 60398.8us
I20260807 19:49:59.946439 8910 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 8: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:49:59.953472 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.958005 12905 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:49:59.960711 12927 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:49:59.963462 12921 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:49:59.968798 12927 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:49:59.969271 12927 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:49:59.969447 13058 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.971381 13101 consensus_queue.cc:1059] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:49:59.973090 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:49:59.935878 12991 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.983155 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.983971 9017 raft_consensus.cc:2368] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:49:59.985186 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.989776 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:59.990465 9017 raft_consensus.cc:2376] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7. Re-sending same reply.
I20260807 19:49:59.979204 13023 consensus_queue.cc:1059] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260807 19:49:59.992146 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.992847 9021 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:49:59.994055 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.994313 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.994710 13051 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:49:59.995267 13051 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:49:59.994059 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.996032 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:49:59.990088 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:49:59.995658 12943 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260807 19:49:59.995645 9019 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:49:59.999008 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:49:59.972131 13065 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260807 19:49:59.973948 12905 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.000689 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.122975 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:00.123423 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 133418us
W20260807 19:50:00.071560 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 81428.6us
I20260807 19:50:00.124181 8856 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.126430 8856 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
W20260807 19:50:00.073386 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 144514us
I20260807 19:50:00.127409 13069 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:00.128119 13069 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:00.128582 13069 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:00.132138 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.133379 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.134363 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.135258 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.136103 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.137522 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.001120 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:00.036677 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.140769 8903 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.138806 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.126032 8968 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.124553 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.125519 9018 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.143409 8968 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.146302 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.147600 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:00.150084 8740 connection.cc:256] RPC call timeout handler was delayed by -1.7365221460349858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260807 19:50:00.150729 8740 connection.cc:256] RPC call timeout handler was delayed by -1.7257221459876746s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260807 19:50:00.151134 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:00.153014 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.154165 8903 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:00.154486 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:00.155913 8740 connection.cc:256] RPC call timeout handler was delayed by -1.6942992049735039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
I20260807 19:50:00.155265 13040 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:00.159248 13051 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:00.013118 13010 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:00.160482 8968 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.161019 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.161144 12609 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:00.161556 12775 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
W20260807 19:50:00.160598 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 4ec2edf1da424177aaad8b93086118b1 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:50:00.162160 13071 consensus_queue.cc:1048] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:00.026161 13102 consensus_queue.cc:1048] T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.168s
I20260807 19:50:00.164706 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.180229 13069 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:00.166747 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.161687 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.189635 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.190275 9023 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:00.191094 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.191975 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.171115 9018 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:50:00.000551 12965 raft_consensus.cc:3055] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
W20260807 19:50:00.173139 8740 connection.cc:256] RPC call timeout handler was delayed by -1.6294221461284906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260807 19:50:00.192857 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:00.192984 8740 connection.cc:256] RPC call timeout handler was delayed by -1.3223439210560173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260807 19:50:00.193137 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.193732 9023 raft_consensus.cc:2410] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:50:00.178930 12905 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.174113 13040 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:00.194435 13040 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.179339 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.195407 13059 raft_consensus.cc:3055] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Stepping down as leader of term 1
I20260807 19:50:00.195901 13059 raft_consensus.cc:740] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:00.196063 13069 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:00.196473 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.197535 9027 raft_consensus.cc:2410] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 19:50:00.197728 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.198364 9023 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:00.198487 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.199240 9027 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:00.199244 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.200285 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.201402 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.202517 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.203224 8913 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:00.203253 13040 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.204511 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.205410 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.205921 8913 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:50:00.203836 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.206851 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.208209 8913 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:00.209364 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.178766 12991 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.183183 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.139899 12950 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:00.143002 13038 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:00.194085 9018 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:50:00.207312 8904 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:00.216187 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.213063 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.217247 8913 raft_consensus.cc:2410] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:50:00.213972 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.213075 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.214828 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.218757 8783 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:00.218641 8776 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:00.218266 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.219736 9022 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:50:00.220099 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.185276 13051 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:00.220793 8904 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:00.238893 9027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8977afab19724738a872c7880e363557"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:00.239283 13051 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:00.239739 13051 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:00.240089 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:00.240741 8904 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.239583 9027 raft_consensus.cc:606] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:50:00.241814 9027 raft_consensus.cc:614] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:50:00.242537 12919 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:00.243130 12919 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:00.240751 12681 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:50:00.245433 12919 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:00.183699 13010 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.248142 13010 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.221602 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.217396 8778 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:50:00.223172 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.223989 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:00.228801 13059 consensus_queue.cc:260] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.229272 12927 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.236981 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.238528 13038 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:00.249414 8904 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:00.251343 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.255669 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.258373 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.258562 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.185633 12846 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260807 19:50:00.259713 12927 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:00.181713 13025 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.260660 9021 raft_consensus.cc:2368] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:00.260982 13038 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.261894 13025 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.262296 13059 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:50:00.262928 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:00.263540 9019 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:00.213191 12921 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:00.205916 13040 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:00.217798 13012 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:00.213433 12965 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:00.272183 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:00.269726 8740 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:00.273576 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet 8977afab19724738a872c7880e363557 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:50:00.274528 8783 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:50:00.213514 12950 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:00.277705 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.280499 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.281086 8904 raft_consensus.cc:2376] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9. Re-sending same reply.
I20260807 19:50:00.282294 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.278637 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.283509 8788 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
I20260807 19:50:00.279102 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.285040 8776 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
I20260807 19:50:00.285243 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.286733 8913 raft_consensus.cc:2393] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:00.292089 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.294466 8740 leader_election.cc:400] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:00.295521 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.280249 8778 raft_consensus.cc:2376] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6. Re-sending same reply.
I20260807 19:50:00.298166 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:00.300213 8913 raft_consensus.cc:2393] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:00.307087 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:00.307873 8904 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.286268 12950 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:00.272552 12965 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.290025 12775 raft_consensus.cc:2749] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260807 19:50:00.264572 12921 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.297683 13058 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.301405 12983 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:00.331058 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.280726 13025 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260807 19:50:00.335882 9018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:00.337329 9018 raft_consensus.cc:606] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:00.305992 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:00.340819 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:00.343506 8904 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:00.342694 13074 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.331439 13093 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:00.349660 12775 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:50:00.349346 13041 raft_consensus.cc:3055] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:00.356186 13041 raft_consensus.cc:740] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:00.372099 9020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1587) took 1507 ms. Trace:
I20260807 19:50:00.377063 9020 rpcz_store.cc:276] 0807 19:49:58.859262 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:59.190892 (+331630us) service_pool.cc:224] Handling call
0807 19:49:59.270225 (+ 79333us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:49:59.270447 (+ 222us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:59.397575 (+127128us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:59.397608 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:59.397652 (+ 44us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:49:59.397698 (+ 46us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:00.366569 (+968871us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540006c2d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:00.366628 (+ 59us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:00.366783 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296359168}
I20260807 19:50:00.395483 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:00.365512 12609 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:00.397274 12609 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.398386 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:50:00.410178 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 71990.6us
I20260807 19:50:00.394582 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:00.414944 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:00.417825 8740 leader_election.cc:400] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:50:00.418124 8740 leader_election.cc:403] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260807 19:50:00.420883 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:50:00.424197 8614 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 300f918f185249e6895ab57edd060d31 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 7
I20260807 19:50:00.416311 12983 consensus_queue.cc:1048] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:00.430862 8853 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.432224 8740 leader_election.cc:400] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:50:00.419301 13025 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260807 19:50:00.449899 13072 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.796s user 0.002s sys 0.002s
I20260807 19:50:00.419317 13074 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260807 19:50:00.452612 13051 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.454989 13051 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:00.415002 13093 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.042s
I20260807 19:50:00.457841 12921 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.415403 13041 consensus_queue.cc:260] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.430646 13140 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Leader election lost for term 6. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:50:00.459407 13041 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:00.468597 8968 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.477308 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:00.478307 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 47539.2us
I20260807 19:50:00.434286 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:00.480882 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 46683.4us
I20260807 19:50:00.481720 8740 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.484061 8788 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:50:00.486294 13140 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:00.486536 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.495422 8783 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:00.488211 8787 raft_consensus.cc:1240] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:00.482829 13065 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:00.500598 13065 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:00.501082 13065 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:00.490674 13038 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.505168 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.490849 12921 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:50:00.505822 8783 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.491678 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.491706 12882 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:00.507171 12882 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:00.507596 12882 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:00.505726 8906 raft_consensus.cc:1275] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:50:00.522841 8779 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:00.528379 12965 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:00.531304 8903 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:50:00.528419 8740 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.543210 13140 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:00.543686 8737 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.545570 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:00.548995 8915 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53310 (request call id 357) took 1327 ms (client timeout 1765 ms). Trace:
W20260807 19:50:00.549496 8915 rpcz_store.cc:269] 0807 19:49:59.221667 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:59.222190 (+ 523us) service_pool.cc:224] Handling call
0807 19:50:00.548781 (+1326591us) spinlock_profiling.cc:243] Waited 383 ms on lock 0x7b54000a3a54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:00.548914 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":881967872}
I20260807 19:50:00.553475 12921 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:00.554122 12921 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:00.554517 12921 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:00.483422 12950 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:00.561578 8915 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:00.564038 13045 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:50:00.565201 13045 raft_consensus.cc:3055] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:00.565588 13045 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:00.566664 13045 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:00.567929 13045 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:00.569242 13140 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.570890 8902 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:00.572556 13065 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.580793 12950 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:00.582451 8856 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.583427 8972 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:00.585198 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:00.586418 13059 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.585438 13010 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:00.587383 13010 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:00.587785 13010 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.580816 12882 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.591751 8789 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.594592 13065 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:00.599673 8972 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:00.601107 8972 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:00.604717 12882 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.604897 12921 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.606932 12921 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:00.602656 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.609195 13125 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:00.609930 13125 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:00.610723 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:00.611135 8902 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:50:00.600172 9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37380 (request call id 279) took 1364 ms. Trace:
I20260807 19:50:00.610291 13125 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:00.616503 9008 rpcz_store.cc:276] 0807 19:49:59.235810 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:59.268225 (+ 32415us) service_pool.cc:224] Handling call
0807 19:49:59.987412 (+719187us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:49:59.987629 (+ 217us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:49:59.987663 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:49:59.987710 (+ 47us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:49:59.987758 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:49:59.987814 (+ 56us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:00.599991 (+612177us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b540006b3d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:00.600109 (+ 118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646739200}
I20260807 19:50:00.617370 8789 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:00.612922 8856 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.619417 12998 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:50:00.621678 8856 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.623168 13038 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:50:00.623437 8856 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.616623 13125 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:00.627319 8629 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 3 to 7, VOTER 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) evicted. New cstate: current_term: 6 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:50:00.612409 12965 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:00.601487 8737 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.637483 8902 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:00.639115 8779 raft_consensus.cc:1275] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:50:00.629592 8629 catalog_manager.cc:5697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 5 to 7, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 7 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:50:00.641407 8783 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:00.641700 8972 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.643668 8740 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.644474 8972 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:00.645159 8740 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:00.645598 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:00.646940 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:00.654039 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:00.654358 8740 leader_election.cc:400] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:00.655777 8740 leader_election.cc:403] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260807 19:50:00.660842 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:00.668326 8972 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:00.716027 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:00.685902 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:00.722347 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 68137.3us
I20260807 19:50:00.723920 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.724181 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.640697 12965 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:00.647794 13140 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.647825 12882 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:00.740227 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.729359 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.753306 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.761231 8789 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:00.665191 12921 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:00.647938 13040 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:00.704221 13059 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:00.704978 13041 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:00.700897 13065 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:00.738032 13093 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:00.717314 13124 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:00.788417 8790 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:00.726500 13054 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:00.801857 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.804531 8788 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:00.751914 12965 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:00.773707 8780 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:00.795392 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:00.799294 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 70568us
I20260807 19:50:00.802307 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.808723 8779 raft_consensus.cc:2368] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:00.815027 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.828971 8778 raft_consensus.cc:2393] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 4: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:00.751688 13012 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
W20260807 19:50:00.823954 9838 leader_election.cc:336] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.608s (SENT)
I20260807 19:50:00.816351 12775 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:00.817831 13040 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260807 19:50:00.841562 13137 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.131s user 0.001s sys 0.000s
I20260807 19:50:00.843890 9018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:00.844141 13041 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:00.791271 12921 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:00.882540 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.883232 8789 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:00.884091 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:00.856590 13155 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.077s user 0.000s sys 0.001s
I20260807 19:50:00.857438 13054 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:00.903277 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.904052 8789 raft_consensus.cc:2368] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6.
I20260807 19:50:00.859133 13086 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:00.873060 8779 raft_consensus.cc:1275] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:00.876394 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.877594 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.879047 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.791499 13071 consensus_queue.cc:1048] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260807 19:50:00.886854 13010 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.894239 8913 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 19:50:00.904860 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.906453 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38436 (request call id 197) took 6948 ms. Trace:
I20260807 19:50:00.907774 8788 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
W20260807 19:50:00.977660 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 265727us
I20260807 19:50:00.980813 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:00.818686 13124 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:00.981769 8788 raft_consensus.cc:2368] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:00.982676 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.983973 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.990859 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.991392 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.991477 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1594) took 1029 ms. Trace:
I20260807 19:50:00.994843 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.995185 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.996335 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.872613 12846 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:01.011813 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.900367 12681 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:01.002197 9017 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260807 19:50:01.031127 9017 pending_rounds.cc:85] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Aborting all ops after (but not including) 3
I20260807 19:50:01.031389 9017 pending_rounds.cc:107] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260807 19:50:01.031616 9017 raft_consensus.cc:2889] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:01.002820 9021 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:01.010188 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:00.992051 8901 rpcz_store.cc:276] 0807 19:49:59.961742 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:00.460816 (+499074us) service_pool.cc:224] Handling call
0807 19:50:00.461134 (+ 318us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:50:00.461552 (+ 418us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:00.461577 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:50:00.462165 (+ 588us) log.cc:844] Serialized 52 byte log entry
0807 19:50:00.526215 (+ 64050us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:00.742222 (+216007us) log.cc:844] Serialized 14 byte log entry
0807 19:50:00.752920 (+ 10698us) log.cc:844] Serialized 14 byte log entry
0807 19:50:00.991038 (+238118us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:00.991202 (+ 164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:00.991249 (+ 47us) raft_consensus.cc:1685] finished
0807 19:50:00.991262 (+ 13us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:00.991411 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10168,"prepare.queue_time_us":83363,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":8382,"raft.queue_time_us":286299,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":241,"thread_start_us":516739,"threads_started":3,"wal-append.queue_time_us":60432}
I20260807 19:50:01.033790 8788 raft_consensus.cc:2393] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:50:00.999711 13144 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.091s user 0.000s sys 0.000s
I20260807 19:50:01.035504 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.040119 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.041587 8776 raft_consensus.cc:2368] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:01.028363 8856 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.029808 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.042982 8775 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:00.970471 8630 rpcz_store.cc:276] 0807 19:49:53.933081 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:53.933471 (+ 390us) service_pool.cc:224] Handling call
0807 19:50:00.881661 (+6948190us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:49:59.693855 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:49:59.694465 (+ 610us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:49:59.694504 (+ 39us) write_op.cc:435] Acquired schema lock
0807 19:49:59.694521 (+ 17us) tablet.cc:662] Decoding operations
0807 19:49:59.707080 (+ 12559us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:49:59.707139 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0807 19:49:59.707165 (+ 26us) tablet.cc:685] Acquiring locks for 19 operations
0807 19:49:59.708512 (+ 1347us) tablet.cc:701] Row locks acquired
0807 19:49:59.708529 (+ 17us) write_op.cc:260] PREPARE: finished
0807 19:49:59.708687 (+ 158us) write_op.cc:270] Start()
0807 19:49:59.708812 (+ 125us) write_op.cc:276] Timestamp: P: 1786132199708637 usec, L: 0
0807 19:49:59.708832 (+ 20us) op_driver.cc:348] REPLICATION: starting
0807 19:49:59.709622 (+ 790us) log.cc:844] Serialized 7468 byte log entry
0807 19:50:00.165210 (+455588us) op_driver.cc:464] REPLICATION: finished
0807 19:50:00.357682 (+192472us) write_op.cc:301] APPLY: starting
0807 19:50:00.359950 (+ 2268us) tablet.cc:1367] starting BulkCheckPresence
0807 19:50:00.424573 (+ 64623us) tablet.cc:1370] finished BulkCheckPresence
0807 19:50:00.427888 (+ 3315us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:50:00.436578 (+ 8690us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:50:00.437027 (+ 449us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0807 19:50:00.437086 (+ 59us) write_op.cc:312] APPLY: finished
0807 19:50:00.437563 (+ 477us) log.cc:844] Serialized 131 byte log entry
0807 19:50:00.464445 (+ 26882us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:00.464656 (+ 211us) write_op.cc:454] Released schema lock
0807 19:50:00.464730 (+ 74us) write_op.cc:341] FINISH: Updating metrics
0807 19:50:00.465265 (+ 535us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:00.465292 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":560,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":189698,"apply.run_cpu_time_us":5793,"apply.run_wall_time_us":110002,"num_ops":19,"prepare.queue_time_us":441634,"prepare.run_cpu_time_us":17967,"prepare.run_wall_time_us":119162,"raft.queue_time_us":169087,"raft.run_cpu_time_us":38107,"raft.run_wall_time_us":172135,"replication_time_us":433224,"spinlock_wait_cycles":55168,"thread_start_us":385051,"threads_started":4,"wal-append.queue_time_us":190336}]]}
I20260807 19:50:01.029034 9008 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:01.032253 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:01.059897 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 47579us
W20260807 19:50:01.062417 8911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s user 0.047s sys 0.060s
W20260807 19:50:01.062896 8911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s user 0.048s sys 0.060s
I20260807 19:50:01.061549 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1217) took 1150 ms. Trace:
I20260807 19:50:01.063830 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:01.071096 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.069726 8904 rpcz_store.cc:276] 0807 19:49:59.910922 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:00.345066 (+434144us) service_pool.cc:224] Handling call
0807 19:50:00.345316 (+ 250us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:00.345496 (+ 180us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 19:50:00.642559 (+297063us) log.cc:844] Serialized 14 byte log entry
0807 19:50:01.060526 (+417967us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:01.060584 (+ 58us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:01.060634 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:01.061364 (+ 730us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:01.061488 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5604,"prepare.queue_time_us":206623,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":36,"raft.queue_time_us":97893,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":708600,"threads_started":3,"wal-append.queue_time_us":336395}
I20260807 19:50:00.977725 13086 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.062093 12950 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:01.028239 12846 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:01.108105 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.057017 12998 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260807 19:50:01.116150 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 78917.5us
I20260807 19:50:01.117830 8790 raft_consensus.cc:2368] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:01.118388 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1599) took 1153 ms. Trace:
I20260807 19:50:01.125201 8906 rpcz_store.cc:276] 0807 19:49:59.964692 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:00.513940 (+549248us) service_pool.cc:224] Handling call
0807 19:50:00.514290 (+ 350us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:50:00.514690 (+ 400us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:00.514717 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:50:00.515301 (+ 584us) log.cc:844] Serialized 52 byte log entry
0807 19:50:00.618431 (+103130us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:00.619820 (+ 1389us) log.cc:844] Serialized 14 byte log entry
0807 19:50:00.626763 (+ 6943us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:00.626924 (+ 161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:01.109613 (+482689us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b2400420968. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878384594 00007f687838269b 00007f687853e6d9 00007f687853e365 00007f687853d8c0 00007f687b1cb638 00007f687b1cbd98 00007f687b1a3588 00007f687b19e95a 00007f687b19dda6
0807 19:50:01.110610 (+ 997us) raft_consensus.cc:1685] finished
0807 19:50:01.118130 (+ 7520us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:01.118318 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":791,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":2258,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":4186,"spinlock_wait_cycles":13118848,"thread_start_us":63441,"threads_started":1,"wal-append.queue_time_us":74797}
I20260807 19:50:01.128005 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.135682 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.037811 12921 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.153837 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.037861 13041 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.140300 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.155509 8775 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:01.106583 13159 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.161437 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1600) took 1196 ms. Trace:
I20260807 19:50:01.065609 13054 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.157155 12991 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:01.189504 8910 rpcz_store.cc:276] 0807 19:49:59.964972 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:00.521423 (+556451us) service_pool.cc:224] Handling call
0807 19:50:00.546926 (+ 25503us) raft_consensus.cc:1471] Updating replica for 3 ops
0807 19:50:00.547330 (+ 404us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:00.547358 (+ 28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0807 19:50:00.548073 (+ 715us) log.cc:844] Serialized 78 byte log entry
0807 19:50:00.703090 (+155017us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:00.752649 (+ 49559us) log.cc:844] Serialized 14 byte log entry
0807 19:50:00.986045 (+233396us) log.cc:844] Serialized 14 byte log entry
0807 19:50:00.987890 (+ 1845us) log.cc:844] Serialized 14 byte log entry
0807 19:50:00.988073 (+ 183us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:00.988150 (+ 77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:01.161202 (+173052us) raft_consensus.cc:1685] finished
0807 19:50:01.161224 (+ 22us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:01.161372 (+ 148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74952,"prepare.run_cpu_time_us":142,"prepare.run_wall_time_us":3261,"raft.queue_time_us":165207,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":24175,"thread_start_us":386509,"threads_started":2,"wal-append.queue_time_us":231545}
I20260807 19:50:01.227654 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:01.207243 13146 log.cc:927] Time spent T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.097s user 0.000s sys 0.001s
W20260807 19:50:01.242537 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 83390.2us
I20260807 19:50:01.202809 13124 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:01.246173 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 53564.6us
I20260807 19:50:01.275663 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:01.286478 8908 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:01.287356 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:01.287959 8780 raft_consensus.cc:2376] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5. Re-sending same reply.
I20260807 19:50:01.286567 8910 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260807 19:50:01.294070 13155 log.cc:927] Time spent T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260807 19:50:01.294114 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1590) took 1334 ms. Trace:
I20260807 19:50:01.297638 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:01.297905 13045 consensus_meta.cc:287] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.730s user 0.013s sys 0.001s
I20260807 19:50:01.299926 13101 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:01.300415 13101 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:01.300745 13101 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:01.295588 8911 rpcz_store.cc:276] 0807 19:49:59.960185 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:00.453294 (+493109us) service_pool.cc:224] Handling call
0807 19:50:00.453655 (+ 361us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:00.454044 (+ 389us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:00.454069 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:00.456239 (+ 2170us) log.cc:844] Serialized 26 byte log entry
0807 19:50:01.063417 (+607178us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:50:01.111688 (+ 48271us) log.cc:844] Serialized 14 byte log entry
0807 19:50:01.192524 (+ 80836us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:01.192778 (+ 254us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:01.293884 (+101106us) raft_consensus.cc:1685] finished
0807 19:50:01.293907 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:01.294050 (+ 143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52924,"prepare.queue_time_us":48007,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":7687,"raft.queue_time_us":83959,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":7924,"thread_start_us":606696,"threads_started":1,"wal-append.queue_time_us":774102}
W20260807 19:50:01.251631 13125 consensus_meta.cc:287] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.627s user 0.010s sys 0.002s
W20260807 19:50:01.269196 13154 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.183s user 0.000s sys 0.000s
I20260807 19:50:01.313162 13132 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260807 19:50:01.246783 13010 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:01.327554 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.228070 12991 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.323793 13125 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.348459 8780 raft_consensus.cc:1275] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260807 19:50:01.353994 8783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s user 0.027s sys 0.052s
W20260807 19:50:01.355038 8783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s user 0.027s sys 0.052s
I20260807 19:50:01.359925 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:01.360939 8780 raft_consensus.cc:2368] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:01.367615 8968 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:01.377955 8780 raft_consensus.cc:1275] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:50:01.364624 12846 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:01.397487 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:01.377791 13183 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 19:50:01.393887 12965 consensus_meta.cc:287] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.540s user 0.014s sys 0.002s
I20260807 19:50:01.425351 8780 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:01.425782 12809 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:01.426167 12809 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:01.427277 12809 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:01.382483 13059 consensus_queue.cc:1059] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260807 19:50:01.409775 13172 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:01.413280 13183 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:50:01.450258 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 63720.9us
I20260807 19:50:01.447849 13011 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:01.407439 13071 consensus_queue.cc:1048] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260807 19:50:01.452570 13011 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:01.455160 13011 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:01.461045 13183 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:50:01.455739 13179 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.108s user 0.000s sys 0.001s
I20260807 19:50:01.456535 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:01.465339 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:01.466105 13125 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:01.471565 8902 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:01.489284 12991 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:01.490932 12950 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:01.492692 12950 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:01.499318 12809 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:01.421000 12965 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.502209 9018 raft_consensus.cc:606] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
I20260807 19:50:01.502660 9018 raft_consensus.cc:614] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:50:01.503288 12965 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:01.469697 13172 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.511765 13011 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260807 19:50:01.513526 9839 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:01.514482 13172 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:01.516469 13186 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.065s user 0.001s sys 0.000s
I20260807 19:50:01.518756 12681 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:01.519374 12681 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:01.523236 13102 raft_consensus.cc:993] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:01.523782 13102 raft_consensus.cc:1081] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Signalling peer 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:01.524161 9015 raft_consensus.cc:1275] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260807 19:50:01.526278 13172 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:01.516520 13086 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:01.527426 13172 raft_consensus.cc:3055] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:01.527827 13172 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:01.533315 13172 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.534569 13172 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:01.514957 13158 log.cc:927] Time spent T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.480s user 0.000s sys 0.003s
I20260807 19:50:01.547108 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.536398 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:01.542445 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:01.547751 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:01.553138 8910 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:01.556178 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:01.557126 8908 raft_consensus.cc:2393] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 2: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:01.559808 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:01.561865 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 91312.5us
I20260807 19:50:01.562711 8968 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:01.565367 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:01.566052 8906 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:50:01.572252 8968 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:01.578645 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:01.578912 9838 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:01.581725 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.582696 9020 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:01.588631 8903 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 9: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:01.590675 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:01.594844 8972 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:01.600525 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.602634 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:01.603374 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:01.603540 8849 leader_election.cc:400] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:01.604133 8915 raft_consensus.cc:2368] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:01.562630 12905 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260807 19:50:01.573019 13086 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:01.605000 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.647719 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.666317 8910 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:01.656625 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.647437 8906 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:01.725718 8902 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:01.606249 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:01.725627 9027 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:01.726413 9019 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 2.
I20260807 19:50:01.606545 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.598347 12556 consensus_queue.cc:1048] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.149s
W20260807 19:50:01.747361 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 150294us
W20260807 19:50:01.682333 13188 log.cc:927] Time spent T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.181s user 0.002s sys 0.000s
W20260807 19:50:01.689631 13185 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260807 19:50:01.639060 12921 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:01.599709 13012 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:01.585171 13184 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:50:01.758859 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 153960us
I20260807 19:50:01.669210 13040 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:50:01.756110 8856 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:01.684130 13055 raft_consensus.cc:3055] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:01.605849 12775 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:01.675096 13202 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.639312 13200 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:50:01.790720 13190 log.cc:927] Time spent T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.120s user 0.000s sys 0.001s
W20260807 19:50:01.781351 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 186604us
I20260807 19:50:01.779269 8849 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.633493 13011 consensus_queue.cc:1059] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.142s
I20260807 19:50:01.759742 12927 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:01.763047 12921 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:01.779134 13012 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:01.775635 13162 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.794977 8849 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.784732 13040 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:01.752101 13183 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.786279 13055 raft_consensus.cc:740] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:01.807546 8849 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.790000 13161 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:01.802534 13012 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:01.777153 12681 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.802276 12921 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:50:01.808036 13101 consensus_meta.cc:287] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.507s user 0.013s sys 0.000s
I20260807 19:50:01.808096 13040 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:01.802517 13038 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.796624 13086 raft_consensus.cc:697] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:01.813431 13161 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:01.820546 9022 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:01.793987 12775 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.810597 12991 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:01.804608 13200 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260807 19:50:01.824762 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 63343.3us
I20260807 19:50:01.815526 13172 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.802977 12927 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:01.836477 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 229988us
I20260807 19:50:01.841670 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:01.841674 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet ef1e484df4724d9896a8f46b89923261 (from a38f4d6088cf4c8cb510200ca0555801 to 671d8754170646eaac5748745eba58b4) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:50:01.842682 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260807 19:50:01.844270 13206 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.058s user 0.000s sys 0.001s
I20260807 19:50:01.844089 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.844872 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
I20260807 19:50:01.846514 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.847926 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.852049 9020 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:01.848815 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.856956 9018 raft_consensus.cc:2368] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:01.858053 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.858381 13159 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.859267 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.860168 9018 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:01.844600 12905 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.865233 13183 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:01.832048 13161 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:01.863714 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.867064 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.849949 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.850240 9027 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 2.
W20260807 19:50:01.851198 13203 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260807 19:50:01.871695 13159 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:01.851449 12927 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:01.849005 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:01.844940 13204 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.085s user 0.000s sys 0.001s
I20260807 19:50:01.846705 12681 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:01.864598 13168 raft_consensus.cc:3055] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:01.873119 8849 leader_election.cc:400] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:01.858965 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.872365 9017 raft_consensus.cc:2376] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9. Re-sending same reply.
I20260807 19:50:01.884837 13168 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:01.842589 12950 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.885780 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.886535 9017 raft_consensus.cc:2368] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:01.887399 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.888463 9019 raft_consensus.cc:2368] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:01.879415 9020 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
W20260807 19:50:01.888156 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 49589.9us
I20260807 19:50:01.890779 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.892792 8849 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.873904 8856 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:01.848210 13205 log.cc:927] Time spent T 10901e14a5e8471291ad1d7c203b6740 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260807 19:50:01.899544 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.901077 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.832739 13200 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:01.902607 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:01.903916 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.904799 9019 raft_consensus.cc:2368] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:50:01.904902 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.907938 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.911612 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.912109 9022 raft_consensus.cc:2376] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6. Re-sending same reply.
I20260807 19:50:01.884938 9890 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:50:01.914148 8849 leader_election.cc:400] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:01.916117 8849 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.917717 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:50:01.919214 8849 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:01.906629 13162 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:01.911067 13172 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:50:01.906306 12950 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:01.908094 13200 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:01.931813 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:01.940563 13219 raft_consensus.cc:2764] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260807 19:50:01.937871 12991 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:50:01.975826 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 56675.9us
I20260807 19:50:01.822039 13101 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.931303 13172 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:01.912549 13055 consensus_queue.cc:260] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:01.986833 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 414661us
W20260807 19:50:01.987854 13208 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.162s user 0.000s sys 0.001s
I20260807 19:50:02.047871 13218 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.048641 13218 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.055123 13200 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.057775 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.058343 9019 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:01.988615 13215 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 19:50:01.991582 13214 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.098s user 0.001s sys 0.000s
I20260807 19:50:01.885814 13168 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.997210 12775 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:01.925696 13086 consensus_queue.cc:237] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:01.999143 9017 raft_consensus.cc:2368] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:02.000206 13101 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:01.980638 13170 log.cc:927] Time spent T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.050s user 0.000s sys 0.001s
I20260807 19:50:02.089982 13161 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.091876 9017 raft_consensus.cc:1240] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260807 19:50:02.008996 8849 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.014000 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 72) took 2627 ms. Trace:
I20260807 19:50:02.029796 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.031198 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.031858 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.032707 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.036628 8789 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:02.022274 13172 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:50:02.103960 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:02.104279 9017 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260807 19:50:01.929952 12921 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.059610 8783 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
W20260807 19:50:02.073179 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 47350.2us
I20260807 19:50:02.078768 8632 catalog_manager.cc:5697] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:02.080287 13200 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.089437 13007 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:02.093254 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 84360.4us
I20260807 19:50:02.121747 9015 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260807 19:50:02.123047 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.107152 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.094549 8776 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:50:02.092743 8629 rpcz_store.cc:276] 0807 19:49:59.386546 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:59.386750 (+ 204us) service_pool.cc:224] Handling call
0807 19:50:02.013922 (+2627172us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:50:01.197386 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:50:01.197987 (+ 601us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:50:01.198026 (+ 39us) write_op.cc:435] Acquired schema lock
0807 19:50:01.198044 (+ 18us) tablet.cc:662] Decoding operations
0807 19:50:01.198437 (+ 393us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:50:01.198503 (+ 66us) write_op.cc:641] Partition lock acquired for write op
0807 19:50:01.198530 (+ 27us) tablet.cc:685] Acquiring locks for 2 operations
0807 19:50:01.198858 (+ 328us) tablet.cc:701] Row locks acquired
0807 19:50:01.198876 (+ 18us) write_op.cc:260] PREPARE: finished
0807 19:50:01.199030 (+ 154us) write_op.cc:270] Start()
0807 19:50:01.226032 (+ 27002us) write_op.cc:276] Timestamp: P: 1786132201198976 usec, L: 0
0807 19:50:01.226065 (+ 33us) op_driver.cc:348] REPLICATION: starting
0807 19:50:01.338128 (+112063us) log.cc:844] Serialized 912 byte log entry
0807 19:50:01.702684 (+364556us) op_driver.cc:464] REPLICATION: finished
0807 19:50:01.988433 (+285749us) write_op.cc:301] APPLY: starting
0807 19:50:01.988539 (+ 106us) tablet.cc:1367] starting BulkCheckPresence
0807 19:50:01.988686 (+ 147us) tablet.cc:1370] finished BulkCheckPresence
0807 19:50:01.988807 (+ 121us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:50:01.989112 (+ 305us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:50:01.989335 (+ 223us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 19:50:01.989390 (+ 55us) write_op.cc:312] APPLY: finished
0807 19:50:01.989683 (+ 293us) log.cc:844] Serialized 28 byte log entry
0807 19:50:01.989908 (+ 225us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:01.990025 (+ 117us) write_op.cc:454] Released schema lock
0807 19:50:01.990073 (+ 48us) write_op.cc:341] FINISH: Updating metrics
0807 19:50:01.990567 (+ 494us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:01.990594 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":94097,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":259148,"apply.run_cpu_time_us":2152,"apply.run_wall_time_us":2152,"num_ops":2,"prepare.queue_time_us":360646,"prepare.run_cpu_time_us":3812,"prepare.run_wall_time_us":152727,"replication_time_us":443745,"thread_start_us":409651,"threads_started":2}]]}
I20260807 19:50:02.111091 8856 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:02.127413 8783 raft_consensus.cc:1275] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260807 19:50:02.130169 8780 raft_consensus.cc:2393] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 7: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
W20260807 19:50:02.119896 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 229197us
I20260807 19:50:02.133473 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.114271 8632 catalog_manager.cc:5697] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 reported cstate change: term changed from 2 to 5, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 5 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:50:02.137254 8849 leader_election.cc:400] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:02.137709 8849 leader_election.cc:403] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260807 19:50:02.142285 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.142992 8788 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 1.
I20260807 19:50:02.144625 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.092522 13011 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.109622 12991 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:02.044247 13040 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.118892 8789 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:51024
I20260807 19:50:02.158644 8740 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.130649 12681 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:02.161309 8789 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:02.165858 8789 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:02.163933 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.172935 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:02.173779 13215 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:02.174714 13215 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.175123 13215 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:02.144745 13010 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:02.180531 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.144891 13218 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.182847 9021 raft_consensus.cc:2368] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:50:02.185114 13237 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:02.185352 13218 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:02.166417 13231 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:02.144184 12775 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 19:50:02.144541 13159 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260807 19:50:02.201061 8789 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.141132 13007 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.184453 12991 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:02.216147 8789 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.215438 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.144080 13224 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.213s
W20260807 19:50:02.095746 12905 consensus_queue.cc:1175] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a38f4d6088cf4c8cb510200ca0555801" responder_term: 7 status { last_received { term: 6 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260807 19:50:02.202296 13038 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:02.163393 13011 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.221969 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:02.203071 13231 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.214205 13010 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:02.230338 13038 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.258678 13012 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.259725 13038 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.238371 13231 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:02.260756 13241 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:02.261411 13241 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.262148 13007 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:02.263455 13217 log.cc:927] Time spent T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.271s user 0.001s sys 0.000s
I20260807 19:50:02.223922 13218 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:02.264237 9890 raft_consensus.cc:1240] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:02.243327 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.227737 13229 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.225618 13200 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 19:50:02.359418 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 116156us
I20260807 19:50:02.374883 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.376199 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.378226 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:02.256168 8769 tablet_service.cc:1558] Processing DeleteTablet for tablet 300f918f185249e6895ab57edd060d31 with delete_type TABLET_DATA_TOMBSTONED (TS 8e9ad2b5e2a94d4faff90af735c6d792 not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:60644
I20260807 19:50:02.382227 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:02.263645 13241 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:02.257334 13161 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.220909 13216 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.404s
I20260807 19:50:02.266212 13011 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:02.371528 13218 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:02.379419 13224 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:02.260752 12921 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.382692 9885 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:02.460852 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:02.467362 8908 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:50:02.400938 13237 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:02.372424 13229 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.474674 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:02.482757 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 336446us
I20260807 19:50:02.471737 13012 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:02.523674 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 249567us
I20260807 19:50:02.526042 8970 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:02.527311 13071 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:02.527755 13071 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:02.528111 13071 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:02.530467 13168 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:02.533516 12943 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:02.533918 13055 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:02.543000 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:02.543702 9008 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:02.544735 13240 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.545349 13240 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.547701 13240 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.551612 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:02.542892 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:02.553860 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:02.563026 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:02.480675 13040 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:02.565990 8789 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:02.566687 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:50:02.571816 9838 leader_election.cc:336] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.512s (SENT)
I20260807 19:50:02.577907 8790 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:02.592262 8780 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:50:02.596226 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 220097us
I20260807 19:50:02.597412 12921 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:02.597013 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.597967 12921 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.592801 8968 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:02.599979 8740 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.564062 13200 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.567552 13224 raft_consensus.cc:476] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Not starting pre-election -- already a leader
I20260807 19:50:02.608439 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:02.602782 9023 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:02.602646 8856 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.605417 9008 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
I20260807 19:50:02.607393 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.534130 12943 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.617805 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.615226 8780 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:02.577630 13202 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:02.610548 13132 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 19:50:02.484853 13010 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.626746 8968 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:02.591928 13247 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:02.603683 12775 raft_consensus.cc:2764] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260807 19:50:02.591769 12681 consensus_queue.cc:1059] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260807 19:50:02.629756 13244 consensus_queue.cc:1048] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260807 19:50:02.627841 13202 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.633693 12943 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:02.606508 13218 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:02.619355 13224 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:50:02.623824 13229 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.647858 9027 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:50:02.656172 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 53607.4us
I20260807 19:50:02.658078 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:02.616664 13200 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.653851 13218 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } attrs { replace: true } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:02.664328 13255 log.cc:927] Time spent T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260807 19:50:02.666325 13218 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:02.652081 13224 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.654116 13086 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 19:50:02.667698 13086 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:02.667259 13224 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:02.668152 13086 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:50:02.676167 13224 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.678449 13224 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.620201 13040 consensus_queue.cc:1048] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260807 19:50:02.679276 13011 raft_consensus.cc:3055] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:02.679769 13011 raft_consensus.cc:740] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:02.622881 12927 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260807 19:50:02.687228 13258 tablet_replica.cc:333] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: stopping tablet replica
I20260807 19:50:02.693218 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:50:02.660728 13247 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:02.688495 13258 raft_consensus.cc:2243] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260807 19:50:02.688125 9023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1624) took 1160 ms. Trace:
I20260807 19:50:02.699447 8856 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:02.661403 13219 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:02.682826 13237 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:02.722733 9023 rpcz_store.cc:276] 0807 19:50:01.528128 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:01.528267 (+ 139us) service_pool.cc:224] Handling call
0807 19:50:01.583733 (+ 55466us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800093398. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c63d2 00007f6881432219 00007f6880585640 00007f688064debb 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:50:01.583991 (+ 258us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:02.609479 (+1025488us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:02.687861 (+ 78382us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54002945d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:02.688061 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2372430080}
I20260807 19:50:02.732890 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:02.719383 13224 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:02.728469 12998 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:02.722803 13247 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:02.742710 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 79381.2us
W20260807 19:50:02.747026 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 129306us
I20260807 19:50:02.747948 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:02.764681 9018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s user 0.000s sys 0.025s
W20260807 19:50:02.785169 9018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s user 0.005s sys 0.030s
I20260807 19:50:02.737623 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:02.739656 12998 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:02.736500 13229 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:02.739931 13219 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260807 19:50:02.815307 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 68984.7us
W20260807 19:50:02.817117 8849 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s user 0.008s sys 0.027s
W20260807 19:50:02.817517 8849 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s user 0.008s sys 0.027s
W20260807 19:50:02.818121 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 681022us
I20260807 19:50:02.819373 8849 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.740096 12921 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.815191 8740 leader_election.cc:400] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:02.824713 8740 leader_election.cc:403] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260807 19:50:02.748790 13247 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:02.750093 13202 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:02.788041 13237 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:02.741931 13159 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260807 19:50:02.808357 12998 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:02.742564 13224 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:02.840166 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 112157us
I20260807 19:50:02.821602 13219 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:02.815191 13101 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:02.740391 13011 consensus_queue.cc:260] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.832171 13253 raft_consensus.cc:2749] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:02.830992 12921 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:02.821069 13258 raft_consensus.cc:2272] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260807 19:50:02.824922 13161 consensus_queue.cc:1048] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260807 19:50:02.833038 12775 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260807 19:50:02.838516 13237 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.746637 12927 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260807 19:50:02.786432 13065 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:02.901974 13237 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:02.871431 13216 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:02.874665 13258 ts_tablet_manager.cc:1916] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260807 19:50:02.875284 9838 leader_election.cc:336] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.765s (SENT)
I20260807 19:50:02.910272 9838 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:02.854053 13219 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:50:02.879802 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 53276.4us
I20260807 19:50:02.856879 13159 consensus_queue.cc:1243] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer a38f4d6088cf4c8cb510200ca0555801 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260807 19:50:02.902046 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 77931.5us
I20260807 19:50:02.856091 13229 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:02.854820 13261 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260807 19:50:02.871177 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:02.914592 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.918435 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.915062 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.915355 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.879904 13200 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:02.915819 8790 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 19:50:02.916698 8776 raft_consensus.cc:1275] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:50:02.922567 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.919561 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.923672 8789 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:50:02.919862 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.924141 13229 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.920126 8775 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
I20260807 19:50:02.857915 13101 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.910014 13216 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:02.928718 13216 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.929307 13224 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:02.930095 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 54862.2us
I20260807 19:50:02.930958 13007 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:02.931521 13071 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.933120 13261 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:02.933555 13261 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:02.934435 13261 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.936452 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 1224) took 1079 ms. Trace:
I20260807 19:50:02.921766 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.937623 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.937829 8779 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:02.938350 8789 raft_consensus.cc:2368] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:50:02.939235 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.939918 8776 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 1.
I20260807 19:50:02.937208 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.941479 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.905675 13277 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:02.935631 13261 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:02.942821 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.936843 9019 rpcz_store.cc:276] 0807 19:50:01.856908 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.059158 (+202250us) service_pool.cc:224] Handling call
0807 19:50:02.059464 (+ 306us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:02.085060 (+ 25596us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 19:50:02.171767 (+ 86707us) log.cc:844] Serialized 14 byte log entry
0807 19:50:02.744929 (+573162us) log.cc:844] Serialized 14 byte log entry
0807 19:50:02.814263 (+ 69334us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:02.814818 (+ 555us) log.cc:844] Serialized 26 byte log entry
0807 19:50:02.814996 (+ 178us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:02.815029 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:02.815122 (+ 93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:02.936206 (+121084us) raft_consensus.cc:1685] finished
0807 19:50:02.936226 (+ 20us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:02.936388 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154407,"prepare.queue_time_us":44287,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":1993,"raft.queue_time_us":147384,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":58453,"spinlock_wait_cycles":15708288,"thread_start_us":540737,"threads_started":2,"wal-append.queue_time_us":644236}
I20260807 19:50:02.945184 8776 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:02.935374 13086 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:02.935724 13172 consensus_meta.cc:287] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.807s user 0.009s sys 0.003s
I20260807 19:50:02.945860 9015 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
W20260807 19:50:02.942263 13266 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.192s user 0.001s sys 0.000s
I20260807 19:50:02.946822 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.947296 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.948668 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:02.949807 13269 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.129s user 0.000s sys 0.001s
I20260807 19:50:02.950292 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.958047 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.962913 8775 raft_consensus.cc:2368] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6.
I20260807 19:50:02.946317 13277 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.954584 12998 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:02.958600 8790 raft_consensus.cc:2393] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:02.959877 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.960469 8779 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 9.
I20260807 19:50:02.915817 12905 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260807 19:50:02.968636 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.968779 13159 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.975380 8902 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:02.977591 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:02.973537 13159 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:02.984159 8911 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:02.978536 9885 raft_consensus.cc:1813] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Handling vote request from an unknown peer 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:02.976847 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:02.963930 13240 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:02.986486 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:02.964970 13216 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:02.987185 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:02.989420 8911 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:02.991194 13264 log.cc:927] Time spent T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.303s user 0.000s sys 0.004s
I20260807 19:50:02.991364 9885 raft_consensus.cc:2368] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:02.991602 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.992501 8778 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:02.996378 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:02.997742 8779 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:02.998579 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:03.005903 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:03.008096 13277 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:03.008499 13159 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:02.991185 12998 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.009825 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:03.010568 9890 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:03.012925 8853 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:50:02.965152 13172 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:03.022214 13215 consensus_meta.cc:287] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.847s user 0.009s sys 0.005s
I20260807 19:50:03.034165 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.036625 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.046334 8902 raft_consensus.cc:2393] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:03.026345 13159 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:02.995311 13240 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:03.052261 13231 consensus_meta.cc:287] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.791s user 0.013s sys 0.000s
I20260807 19:50:03.068082 13162 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:03.045950 13215 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.135210 8911 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
W20260807 19:50:03.164091 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:03.196267 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:03.072582 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:03.198930 8776 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 1 FOLLOWER]: Advancing to term 3
I20260807 19:50:03.194864 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:03.200278 8903 raft_consensus.cc:3060] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:03.133123 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:03.080364 8849 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:03.205037 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 124820us
W20260807 19:50:03.207075 8849 leader_election.cc:336] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.683s (SENT)
I20260807 19:50:03.209487 8849 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:03.074594 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:03.149622 9838 leader_election.cc:336] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.944s (SENT)
I20260807 19:50:03.197930 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:50:03.053547 13159 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:50:03.211428 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 61899.8us
W20260807 19:50:03.122725 13168 consensus_meta.cc:287] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.592s user 0.009s sys 0.001s
I20260807 19:50:03.213394 8778 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:03.221506 8849 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:03.211490 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:03.128113 13055 consensus_meta.cc:287] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.594s user 0.013s sys 0.000s
I20260807 19:50:03.123294 13071 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:03.208994 13237 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:03.219417 13279 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.302s user 0.000s sys 0.001s
I20260807 19:50:03.222894 8740 leader_election.cc:400] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:03.111937 13229 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:03.132704 13240 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.240175 9020 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:03.224929 8776 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:50:03.223745 9838 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:03.076538 13277 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:03.209757 12681 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260807 19:50:03.230814 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:03.222415 12998 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:03.232545 13215 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.240108 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.123888 13086 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:03.247992 9838 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:03.251255 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:03.221096 13159 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:03.254743 8740 leader_election.cc:400] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:03.215863 13247 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.262568 8910 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:03.225854 8849 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:03.263522 8903 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:03.270625 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.223822 12775 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:03.265578 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:03.229759 13168 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:03.270960 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:03.283479 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:50:03.283915 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 58152.1us
I20260807 19:50:03.283983 8915 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:03.252493 13285 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:03.208304 13231 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.208530 13162 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.257102 13218 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:03.227191 13007 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260807 19:50:03.227051 13012 consensus_queue.cc:1048] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260807 19:50:03.296262 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.296895 8915 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:03.288872 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:03.297230 8906 raft_consensus.cc:2410] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 19:50:03.297614 8913 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:03.298148 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.298717 8906 raft_consensus.cc:1813] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Handling vote request from an unknown peer 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:03.289500 8740 leader_election.cc:400] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:03.299142 8906 raft_consensus.cc:2368] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:03.290597 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:03.285761 13215 raft_consensus.cc:2764] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260807 19:50:03.258112 13277 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.293253 13219 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.302021 8908 raft_consensus.cc:2393] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:03.302522 8911 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:03.303969 13285 raft_consensus.cc:2804] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:03.304554 13285 raft_consensus.cc:697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [term 9 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:03.305531 13285 consensus_queue.cc:237] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:03.312927 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 496815us
I20260807 19:50:03.313961 8968 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:03.285356 8849 leader_election.cc:400] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:50:03.327090 8849 leader_election.cc:403] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 19:50:03.289229 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:03.328480 8903 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:03.295215 13253 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:03.287861 13010 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:03.338932 13010 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:03.339366 13010 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:50:03.343307 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 58080.3us
I20260807 19:50:03.345786 13207 raft_consensus.cc:2804] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:50:03.252804 13229 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.353193 8911 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:03.356935 8849 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:03.362615 8913 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:50:03.370899 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:03.343807 12846 consensus_queue.cc:1059] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.030s
I20260807 19:50:03.376319 13207 raft_consensus.cc:697] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:03.343734 12681 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 4. Current term is 5.
I20260807 19:50:03.377658 13207 consensus_queue.cc:237] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.378988 13231 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.354782 13059 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:03.381557 13059 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:03.382578 13059 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.383805 13059 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:03.377655 13229 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.386876 13277 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.288646 12775 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:03.389458 13247 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.389765 12775 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.349102 13219 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.390084 13162 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:03.390604 13237 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:05.862710 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 2.57318e+06us
W20260807 19:50:05.926193 13307 log.cc:927] Time spent T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 2.632s user 0.018s sys 0.006s
W20260807 19:50:06.069108 8910 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1656) took 3091 ms (client timeout 1812 ms). Trace:
W20260807 19:50:06.069706 8910 rpcz_store.cc:269] 0807 19:50:02.977211 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.977330 (+ 119us) service_pool.cc:224] Handling call
0807 19:50:06.068630 (+3091300us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b54002fc454. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:06.068824 (+ 194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571259648}
I20260807 19:50:06.071343 8910 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:06.071730 8910 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:05.864336 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:07.002455 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:07.006054 8910 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:07.015319 8910 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:07.029609 8689 debug-util.cc:398] Leaking SignalData structure 0x7b08002f7b60 after lost signal to thread 6318
I20260807 19:50:07.064240 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:50:07.092190 8910 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260807 19:50:07.093222 8910 pending_rounds.cc:85] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 5
I20260807 19:50:07.093717 8910 pending_rounds.cc:107] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260807 19:50:07.094220 8910 raft_consensus.cc:2889] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:07.096247 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1674) took 4058 ms. Trace:
I20260807 19:50:07.096853 8910 rpcz_store.cc:276] 0807 19:50:03.037754 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:06.070821 (+3033067us) service_pool.cc:224] Handling call
0807 19:50:06.071195 (+ 374us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:07.095336 (+1024141us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:07.095797 (+ 461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:07.100540 8910 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260807 19:50:07.101447 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1233) took 4020 ms. Trace:
I20260807 19:50:07.101905 8910 rpcz_store.cc:276] 0807 19:50:03.081139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:07.099132 (+4017993us) service_pool.cc:224] Handling call
0807 19:50:07.099837 (+ 705us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:07.101044 (+ 1207us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:07.101343 (+ 299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:07.104029 8910 raft_consensus.cc:1240] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:07.104823 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1238) took 3968 ms. Trace:
I20260807 19:50:07.105307 8910 rpcz_store.cc:276] 0807 19:50:03.136855 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:07.102948 (+3966093us) service_pool.cc:224] Handling call
0807 19:50:07.103552 (+ 604us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:07.104512 (+ 960us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:07.104738 (+ 226us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:50:07.211933 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:07.670315 13159 consensus_meta.cc:287] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 4.396s user 0.014s sys 0.028s
I20260807 19:50:07.670950 13159 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:07.697094 13283 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 4.819s user 0.018s sys 0.028s
I20260807 19:50:07.707938 13308 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:07.775882 13308 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:07.777159 13159 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:07.798965 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:07.800248 9027 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:07.804913 13172 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:07.825359 8776 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
W20260807 19:50:07.838058 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 1.97352e+06us
I20260807 19:50:07.839433 13237 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:07.831030 13231 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:07.845151 13231 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:07.845175 13207 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.452s user 0.008s sys 0.034s
W20260807 19:50:07.851904 13207 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.459s user 0.008s sys 0.035s
I20260807 19:50:07.839737 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:07.856189 13059 consensus_meta.cc:287] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 4.472s user 0.020s sys 0.031s
I20260807 19:50:07.852464 13308 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:07.866755 13301 log.cc:927] Time spent T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 4.575s user 0.008s sys 0.024s
I20260807 19:50:07.867372 13308 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:07.858717 13237 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:07.869010 13305 log.cc:927] Time spent T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 4.577s user 0.012s sys 0.020s
W20260807 19:50:07.864492 13285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.487s user 0.014s sys 0.013s
W20260807 19:50:07.873315 13285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.496s user 0.015s sys 0.013s
I20260807 19:50:07.873610 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:07.874614 8776 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:07.875430 13285 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:07.876291 13285 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:07.886936 8776 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:07.888432 8775 raft_consensus.cc:1275] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:07.889098 13237 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:07.889719 13237 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:07.890493 13237 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:07.894090 13285 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:07.868336 13308 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:07.888417 8913 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:07.897492 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1243) took 4692 ms. Trace:
W20260807 19:50:07.898372 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 4.98538e+06us
I20260807 19:50:07.897914 8913 rpcz_store.cc:276] 0807 19:50:03.204610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:07.887047 (+4682437us) service_pool.cc:224] Handling call
0807 19:50:07.887581 (+ 534us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:07.897146 (+ 9565us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:07.897407 (+ 261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:07.880894 13231 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:07.906080 13231 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:07.892783 8906 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:50:03.361289 8903 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
W20260807 19:50:07.906831 8972 connection.cc:256] RPC call timeout handler was delayed by -4.9790842731017619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
W20260807 19:50:07.910961 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:07.911098 8903 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 1236) took 4776 ms (client timeout 2217 ms). Trace:
W20260807 19:50:07.912562 8903 rpcz_store.cc:269] 0807 19:50:03.134692 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.286800 (+152108us) service_pool.cc:224] Handling call
0807 19:50:07.910849 (+4624049us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:07.918273 13237 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:07.920096 13041 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 4.567s user 0.055s sys 0.144s
W20260807 19:50:07.921123 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 81598.9us
I20260807 19:50:07.922260 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:07.924932 13308 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:07.934105 13041 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 4.581s user 0.057s sys 0.144s
I20260807 19:50:07.944221 13237 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:07.945143 13237 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:07.945772 8908 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1672) took 4914 ms (client timeout 1728 ms). Trace:
I20260807 19:50:07.953019 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:07.954661 8780 raft_consensus.cc:2393] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:07.955263 8740 leader_election.cc:400] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:50:07.958196 8780 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:07.958633 8783 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:50:07.959718 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:07.968196 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:07.971277 8740 leader_election.cc:400] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
W20260807 19:50:07.950484 8908 rpcz_store.cc:269] 0807 19:50:03.031440 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.033871 (+ 2431us) service_pool.cc:224] Handling call
0807 19:50:07.945225 (+4911354us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b5400461ed4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:07.945552 (+ 327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608756480}
I20260807 19:50:07.987649 9898 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33452 (request call id 315) took 5325 ms. Trace:
I20260807 19:50:07.959635 8789 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:07.988171 9898 rpcz_store.cc:276] 0807 19:50:02.661720 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.661908 (+ 188us) service_pool.cc:224] Handling call
0807 19:50:02.677152 (+ 15244us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:02.677527 (+ 375us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:02.677553 (+ 26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:02.678602 (+ 1049us) log.cc:844] Serialized 494 byte log entry
0807 19:50:02.751858 (+ 73256us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:02.751908 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:02.751982 (+ 74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:07.694827 (+4942845us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b440002250c. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878424855 00007f6878425272 00007f687b1af3bb 00007f687b1aac9f 00007f687b19ea5e 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411 00007f687b1a73c9 00007f687b1a7359
0807 19:50:07.986859 (+292032us) raft_consensus.cc:1685] finished
0807 19:50:07.986883 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:07.987400 (+ 517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4729216,"thread_start_us":72868,"threads_started":1,"wal-append.queue_time_us":161001}
I20260807 19:50:08.041797 13101 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.058398 13237 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:08.080626 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 91424.2us
I20260807 19:50:08.090137 13237 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260807 19:50:08.123852 13010 consensus_meta.cc:287] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 4.783s user 0.012s sys 0.043s
I20260807 19:50:08.124670 13010 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.124192 13274 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.129561 13274 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:08.131888 9850 connection.cc:256] RPC call timeout handler was delayed by -3.9995181760750711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260807 19:50:08.132956 13174 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.133659 13176 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:08.143959 13293 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 4.854s user 0.019s sys 0.017s
W20260807 19:50:08.150368 13289 log.cc:927] Time spent T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 5.229s user 0.004s sys 0.029s
I20260807 19:50:08.143895 13274 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.148926 13174 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.150190 13176 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.160133 13258 ts_tablet_manager.cc:1929] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 6.6
I20260807 19:50:08.162484 13176 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.163807 13174 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.144186 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.164746 9008 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
W20260807 19:50:08.157603 8902 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1654) took 5181 ms (client timeout 2246 ms). Trace:
I20260807 19:50:08.164867 8915 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1670) took 5141 ms. Trace:
W20260807 19:50:08.179562 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 4.82228e+06us
W20260807 19:50:08.170279 8902 rpcz_store.cc:269] 0807 19:50:02.976435 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.976610 (+ 175us) service_pool.cc:224] Handling call
0807 19:50:08.153989 (+5177379us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b54003d77d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.157353 (+ 3364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117654784}
I20260807 19:50:08.185380 13274 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:08.191762 13274 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.160297 9015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37380 (request call id 295) took 5608 ms (client timeout 1858 ms). Trace:
I20260807 19:50:08.175458 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1647) took 5203 ms. Trace:
I20260807 19:50:08.170707 8915 rpcz_store.cc:276] 0807 19:50:03.023364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.345575 (+322211us) service_pool.cc:224] Handling call
0807 19:50:03.345915 (+ 340us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:08.163178 (+4817263us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:08.163228 (+ 50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:08.163282 (+ 54us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:08.163315 (+ 33us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:08.164434 (+ 1119us) spinlock_profiling.cc:243] Waited 4.82 s on lock 0x7b5400360bd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:08.164477 (+ 43us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:08.164681 (+ 204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11077897984}
W20260807 19:50:08.191025 12943 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 4.839s user 0.054s sys 0.108s
W20260807 19:50:08.198695 12943 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 4.847s user 0.054s sys 0.108s
I20260807 19:50:08.199172 13159 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:50:08.192723 9015 rpcz_store.cc:269] 0807 19:50:02.551211 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.551346 (+ 135us) service_pool.cc:224] Handling call
0807 19:50:08.159832 (+5608486us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540006b3d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.160048 (+ 216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810677504}
I20260807 19:50:08.179615 13010 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.166143 13258 log.cc:1199] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: Deleting WAL directory at /tmp/dist-test-task7uuv2e/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786132061667097-6315-0/minicluster-data/ts-0-root/wals/300f918f185249e6895ab57edd060d31
I20260807 19:50:08.210548 13159 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.211007 13161 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.211792 13161 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.210860 12905 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.213230 12905 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.196127 13172 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.214366 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1245) took 5002 ms. Trace:
I20260807 19:50:08.214792 13172 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.214772 8913 rpcz_store.cc:276] 0807 19:50:03.211462 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:07.905856 (+4694394us) service_pool.cc:224] Handling call
0807 19:50:07.906403 (+ 547us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:07.906965 (+ 562us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 19:50:08.030591 (+123626us) log.cc:844] Serialized 14 byte log entry
0807 19:50:08.213787 (+183196us) log.cc:844] Serialized 14 byte log entry
0807 19:50:08.213936 (+ 149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:08.213975 (+ 39us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:08.214062 (+ 87us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:08.214128 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:08.214250 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62309,"thread_start_us":185230,"threads_started":2}
I20260807 19:50:08.193199 8904 rpcz_store.cc:276] 0807 19:50:02.971725 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.971895 (+ 170us) service_pool.cc:224] Handling call
0807 19:50:02.972192 (+ 297us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:03.128667 (+156475us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:03.128698 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:03.128745 (+ 47us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:03.128782 (+ 37us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:08.174817 (+5046035us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54002c4054. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:08.174923 (+ 106us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:08.175224 (+ 301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359338752}
I20260807 19:50:08.192480 13168 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:08.213086 13159 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.212083 13274 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.256896 13274 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:50:08.214232 13161 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.257972 13274 consensus_queue.cc:1059] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:50:08.216709 12905 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.281518 13258 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60644 (request call id 68) took 6258 ms. Trace:
I20260807 19:50:08.282042 13258 rpcz_store.cc:276] 0807 19:50:02.082731 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.162503 (+ 79772us) service_pool.cc:224] Handling call
0807 19:50:02.686813 (+524310us) ts_tablet_manager.cc:1151] Deleting tablet 300f918f185249e6895ab57edd060d31
0807 19:50:02.686915 (+ 102us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0807 19:50:03.380079 (+693164us) tablet_metadata.cc:665] Metadata flushed
0807 19:50:08.159270 (+4779191us) tablet_metadata.cc:665] Metadata flushed
0807 19:50:08.281378 (+122108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9369,"tablet-delete.queue_time_us":303423,"thread_start_us":213810,"threads_started":1}
I20260807 19:50:08.282928 8849 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:08.287103 13168 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.288605 13010 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.288789 13161 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.289273 13010 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.289573 13161 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.291544 13010 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.292636 12905 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.293406 12905 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.281982 12775 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:50:08.297586 12775 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.298246 12775 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.277545 13172 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.281363 13274 consensus_queue.cc:1059] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260807 19:50:08.305819 13274 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.306609 13274 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.354066 9850 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.195:35801 timed out after 1.858s (SENT) [suppressed 39 similar messages]
W20260807 19:50:08.355126 12998 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.970s user 0.028s sys 0.046s
W20260807 19:50:08.372166 12998 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 5.099s user 0.029s sys 0.046s
I20260807 19:50:08.287778 13168 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.292177 13010 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:50:08.372668 12998 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:08.311033 13159 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:08.452200 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 169513us
W20260807 19:50:08.452865 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 335096us
I20260807 19:50:08.467911 13274 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.473843 13161 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.474591 13161 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.475222 13161 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.476611 12905 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.431313 12775 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.430363 13168 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.477238 12905 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.477698 13172 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:08.478293 13010 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.474148 12998 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.500360 8849 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:08.506764 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:08.451692 13277 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.508134 9898 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:08.481673 13168 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.474534 13159 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.513608 13172 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.553808 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:08.569320 12905 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.570649 13172 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.570042 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:08.571655 12905 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:08.572810 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 92702.1us
I20260807 19:50:08.573922 8849 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:08.575078 9885 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33436 (request call id 197) took 5884 ms (client timeout 2961 ms). Trace:
W20260807 19:50:08.575533 9885 rpcz_store.cc:269] 0807 19:50:02.690465 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.717906 (+ 27441us) service_pool.cc:224] Handling call
0807 19:50:08.574614 (+5856708us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b5400750654. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.574868 (+ 254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386916736}
I20260807 19:50:08.576488 13277 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.577546 13161 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.578186 13161 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:08.578809 13277 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.580286 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:08.588376 12905 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:50:08.575647 8849 leader_election.cc:400] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6.
I20260807 19:50:08.591660 8849 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:08.506803 13010 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.598387 12905 raft_consensus.cc:2749] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:08.599645 13010 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.601577 9898 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
W20260807 19:50:08.577487 13265 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 5.891s user 0.017s sys 0.016s
I20260807 19:50:08.593778 13159 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.602618 8849 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:08.538811 13168 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.606324 8853 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:50:08.607417 12905 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:08.607457 13010 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:08.608453 13010 raft_consensus.cc:697] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:08.609473 13051 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.610193 13010 consensus_queue.cc:237] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.611538 12846 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.611770 8849 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:08.612273 12846 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.613372 12905 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260807 19:50:08.614675 13216 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.610188 13051 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.615463 13216 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.617487 13051 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.614818 12846 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.621459 13168 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.626972 13216 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.635617 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:50:08.624326 13038 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260807 19:50:08.647491 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:08.650324 8856 connection.cc:256] RPC call timeout handler was delayed by -2.5652396150399s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
I20260807 19:50:08.651872 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:08.654181 8856 connection.cc:256] RPC call timeout handler was delayed by -1.1955525819212198s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32570000000000005s for a response
I20260807 19:50:08.652622 13162 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.655311 13162 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.656159 13229 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.657114 13229 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.658305 13162 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.659493 13229 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.619987 13011 raft_consensus.cc:3060] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:08.660951 13229 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.661703 13229 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.662811 9020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1220) took 6808 ms (client timeout 2245 ms). Trace:
W20260807 19:50:08.663322 9020 rpcz_store.cc:269] 0807 19:50:01.854936 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:01.907685 (+ 52749us) service_pool.cc:224] Handling call
0807 19:50:08.662412 (+6754727us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540043cbd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.662587 (+ 175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3044964736}
I20260807 19:50:08.664108 13229 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.665244 13229 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.666388 8901 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:08.668627 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1667) took 5674 ms. Trace:
I20260807 19:50:08.668999 8901 rpcz_store.cc:276] 0807 19:50:02.994433 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.332942 (+338509us) service_pool.cc:224] Handling call
0807 19:50:08.665916 (+5332974us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:08.666802 (+ 886us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:08.668409 (+ 1607us) spinlock_profiling.cc:243] Waited 5.33 s on lock 0x7b54003d68d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.668564 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12264015232}
I20260807 19:50:08.608054 13065 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.670786 13065 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.665565 8689 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec820 after lost signal to thread 8695
W20260807 19:50:08.673964 8689 debug-util.cc:398] Leaking SignalData structure 0x7b080007fb80 after lost signal to thread 8828
W20260807 19:50:08.674299 8689 debug-util.cc:398] Leaking SignalData structure 0x7b080007fbc0 after lost signal to thread 8831
I20260807 19:50:08.674988 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:08.675817 9886 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Handling vote request from an unknown peer 671d8754170646eaac5748745eba58b4
I20260807 19:50:08.676611 9886 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 19:50:08.674743 13229 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.680536 13340 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.203s user 0.003s sys 0.001s
I20260807 19:50:08.685300 13229 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:08.686293 8849 connection.cc:256] RPC call timeout handler was delayed by -4.0268846331164241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
W20260807 19:50:08.687381 8849 connection.cc:256] RPC call timeout handler was delayed by -3.9753696788102388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260807 19:50:08.688232 8849 connection.cc:256] RPC call timeout handler was delayed by -3.9090108929667622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2002s for a response
I20260807 19:50:08.688297 13071 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:50:08.689023 8849 connection.cc:256] RPC call timeout handler was delayed by -3.739810893079266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260807 19:50:08.689486 13071 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.690361 8972 connection.cc:256] RPC call timeout handler was delayed by -4.4041799460537732s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260807 19:50:08.690949 8972 connection.cc:256] RPC call timeout handler was delayed by -4.2788563319481909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260807 19:50:08.691249 8972 connection.cc:256] RPC call timeout handler was delayed by -4.0135799460113049s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
W20260807 19:50:08.691517 8972 connection.cc:256] RPC call timeout handler was delayed by -3.746557307895273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260807 19:50:08.691761 8972 connection.cc:256] RPC call timeout handler was delayed by -3.7124264088924974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260807 19:50:08.692030 8972 connection.cc:256] RPC call timeout handler was delayed by -2.7576517099514604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3086s for a response
I20260807 19:50:08.691912 13071 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.696616 13102 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:50:08.697504 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800045c60 after lost signal to thread 8951
W20260807 19:50:08.666106 8911 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 1235) took 5584 ms (client timeout 2019 ms). Trace:
I20260807 19:50:08.692939 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1669) took 5671 ms. Trace:
I20260807 19:50:08.697955 13101 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:08.697059 13071 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:08.698287 13102 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.697238 8972 connection.cc:256] RPC call timeout handler was delayed by -2.5081415420863777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.294s for a response
I20260807 19:50:08.699301 13071 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.700299 8970 connection.cc:256] RPC call timeout handler was delayed by -3.4248363689985126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260807 19:50:08.701550 13102 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.702075 13071 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.703272 12609 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.703940 12609 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.704428 13041 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:08.704874 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.705117 9020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:08.698271 8906 rpcz_store.cc:276] 0807 19:50:03.021931 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.343920 (+321989us) service_pool.cc:224] Handling call
0807 19:50:07.892239 (+4548319us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:07.909622 (+ 17383us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:08.692697 (+783075us) spinlock_profiling.cc:243] Waited 4.55 s on lock 0x7b54002fbf50. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.692842 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10459394944}
I20260807 19:50:08.685423 13241 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:08.705852 9020 raft_consensus.cc:606] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:08.699818 13280 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:08.707535 13280 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.707885 9008 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260807 19:50:08.709672 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:08.702370 12927 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:08.710053 13280 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:08.697672 8968 connection.cc:256] RPC call timeout handler was delayed by -3.0281868809834123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260807 19:50:08.712635 8968 connection.cc:256] RPC call timeout handler was delayed by -2.7905868811067194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260807 19:50:08.710933 8629 catalog_manager.cc:5697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:50:08.704839 13041 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:08.699455 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:08.713557 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:08.714156 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:08.698619 8911 rpcz_store.cc:269] 0807 19:50:03.081672 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.250914 (+169242us) service_pool.cc:224] Handling call
0807 19:50:08.665704 (+5414790us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b54003d68d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:08.665866 (+ 162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87159680}
I20260807 19:50:08.698695 13101 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.715670 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:08.717163 9027 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:50:08.710886 12927 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.718166 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:08.705410 8689 debug-util.cc:398] Leaking SignalData structure 0x7b080039b620 after lost signal to thread 9068
I20260807 19:50:08.718876 9019 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:08.719154 12927 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:08.712956 8968 connection.cc:256] RPC call timeout handler was delayed by -1.9373868810944259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.294s for a response
I20260807 19:50:08.720449 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.716596 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.722429 9015 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
W20260807 19:50:08.719913 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800363220 after lost signal to thread 9937
I20260807 19:50:08.723704 8629 catalog_manager.cc:5697] T ab8e617d098d425cbddc5d78da43ab99 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260807 19:50:08.724711 8689 debug-util.cc:398] Leaking SignalData structure 0x7b0800326e20 after lost signal to thread 9938
W20260807 19:50:08.725761 8689 debug-util.cc:398] Leaking SignalData structure 0x7b080035ff80 after lost signal to thread 13200
I20260807 19:50:08.727005 8783 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:08.728156 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.729363 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.729923 8783 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:08.726847 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:08.731297 8689 debug-util.cc:398] Leaking SignalData structure 0x7b080032a800 after lost signal to thread 13224
I20260807 19:50:08.732023 8776 raft_consensus.cc:1275] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260807 19:50:08.706405 13241 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.735157 13065 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.702179 13102 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.702867 13071 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:08.732223 8689 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4e40 after lost signal to thread 13288
I20260807 19:50:08.737265 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:08.760967 13059 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:50:08.761099 13071 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.730104 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.762068 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.762301 8779 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:08.706240 12609 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.717348 13101 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.696244 13055 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:08.763252 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.736160 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.744879 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.764199 8780 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:08.764539 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.764681 13102 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.745200 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.763826 13055 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.766074 9023 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:50:08.746500 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.767007 9022 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:50:08.746919 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.767462 13055 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.748142 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.768293 13102 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.768517 9018 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:08.768966 13241 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.769991 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:08.770959 9020 raft_consensus.cc:2393] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 3: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:08.749699 8972 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:08.750697 9008 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:08.750890 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.751925 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.753159 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.741578 13041 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.775390 13041 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:08.777369 13055 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:08.778228 13101 raft_consensus.cc:993] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:08.779400 13101 consensus_queue.cc:1048] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:08.780601 13086 raft_consensus.cc:1081] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Signalling peer 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:08.774405 8790 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:08.783677 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:08.784439 9017 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:08.784783 9020 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260807 19:50:08.785907 13274 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.786680 13274 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.754280 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.787938 8778 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:08.755003 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:08.756533 12927 consensus_queue.cc:1059] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:08.788959 8783 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:08.789400 12927 raft_consensus.cc:3055] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:08.789806 12775 raft_consensus.cc:3055] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:08.758714 13280 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:08.790598 13356 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:08.763682 13071 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:08.772146 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1500) took 5700 ms. Trace:
I20260807 19:50:08.774084 8789 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:08.790081 13358 raft_consensus.cc:3055] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
W20260807 19:50:08.785938 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 309279us
I20260807 19:50:08.791189 13356 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.792254 13327 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:50:08.791415 12775 raft_consensus.cc:740] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:08.793560 9017 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:50:08.792703 8740 leader_election.cc:400] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:50:08.794266 13358 raft_consensus.cc:740] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:08.793866 12775 consensus_queue.cc:260] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.795115 8740 leader_election.cc:403] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260807 19:50:08.796022 12775 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:08.795606 13358 consensus_queue.cc:260] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.797313 13358 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:08.798581 13325 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 4. Current term is 5.
I20260807 19:50:08.784961 13342 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:50:08.693161 8849 connection.cc:256] RPC call timeout handler was delayed by -2.8540464348625392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3003s for a response
I20260807 19:50:08.793831 13356 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.794054 13055 raft_consensus.cc:697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:08.804236 8910 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.538s user 0.033s sys 0.079s
I20260807 19:50:08.816391 9019 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
W20260807 19:50:08.819283 8910 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s user 0.033s sys 0.079s
I20260807 19:50:08.789796 12927 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:08.793825 13327 raft_consensus.cc:3055] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:08.829841 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:08.791352 8788 rpcz_store.cc:276] 0807 19:50:03.071370 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:03.071540 (+ 170us) service_pool.cc:224] Handling call
0807 19:50:03.195443 (+123903us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:03.207961 (+ 12518us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:03.207995 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:03.213332 (+ 5337us) log.cc:844] Serialized 26 byte log entry
0807 19:50:07.655868 (+4442536us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b1800114ac0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784e07b7 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a32c1 00007f687b5bfdef
0807 19:50:07.695549 (+ 39681us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:07.796009 (+100460us) log.cc:844] Serialized 14 byte log entry
0807 19:50:07.863494 (+ 67485us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:07.885218 (+ 21724us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:08.771684 (+886466us) raft_consensus.cc:1685] finished
0807 19:50:08.771781 (+ 97us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:08.771930 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25746,"prepare.queue_time_us":927369,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":68,"raft.queue_time_us":130432,"raft.run_cpu_time_us":402,"raft.run_wall_time_us":4712,"spinlock_wait_cycles":9709312,"thread_start_us":166575,"threads_started":3,"wal-append.queue_time_us":5426593}
I20260807 19:50:08.841588 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:08.820971 13274 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:08.855230 8911 raft_consensus.cc:2368] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8.
I20260807 19:50:08.856308 8906 raft_consensus.cc:2393] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:08.856321 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.829349 13327 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:08.859248 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.800276 13342 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:08.865479 13055 consensus_queue.cc:237] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.867370 13342 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:08.868795 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.869663 9020 raft_consensus.cc:2393] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 10: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:08.866950 13065 consensus_queue.cc:1059] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:50:08.828939 13071 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260807 19:50:08.843202 13102 consensus_queue.cc:1048] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260807 19:50:08.871968 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:08.896759 13344 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:08.897614 13344 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:08.900481 8737 connection.cc:256] RPC call timeout handler was delayed by -2.8241826710291207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2961s for a response
I20260807 19:50:08.900913 13102 raft_consensus.cc:3055] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:08.901420 13102 raft_consensus.cc:740] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:08.901106 12927 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.902601 12927 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:08.902437 13102 consensus_queue.cc:260] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.903812 13102 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:08.904735 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:08.906126 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:08.907182 8613 catalog_manager.cc:5028] TS 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): tablet 300f918f185249e6895ab57edd060d31 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:50:08.908310 8913 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260807 19:50:08.909130 13071 raft_consensus.cc:3055] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:08.909510 13071 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:08.910485 13071 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.910656 13327 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:08.912015 13071 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:08.856731 13280 consensus_queue.cc:1059] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
W20260807 19:50:08.913106 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 55956us
I20260807 19:50:08.913683 13101 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:50:08.914402 12681 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:08.908500 8736 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s user 0.057s sys 0.103s
I20260807 19:50:08.918707 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:08.920246 8736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s user 0.058s sys 0.103s
I20260807 19:50:08.920482 9019 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 9
W20260807 19:50:08.926447 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 572436us
I20260807 19:50:08.912056 13327 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:08.945012 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.946900 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.947654 9023 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:08.944880 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.967108 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.967353 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.951342 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.900226 13344 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:08.948329 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 246816us
I20260807 19:50:08.996836 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.002385 8911 raft_consensus.cc:1240] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:09.005033 9019 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
I20260807 19:50:09.012512 9015 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:09.013207 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:08.941761 12775 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:09.020287 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.021003 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.020869 13038 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:09.021529 9850 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:08.929651 12681 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.022341 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.023111 9022 raft_consensus.cc:2393] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:09.023845 13321 raft_consensus.cc:2764] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260807 19:50:09.025192 13321 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:09.025527 13321 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:09.025869 13321 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:09.026717 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 58305.1us
I20260807 19:50:09.026075 12609 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 4 to 6, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:50:09.014635 8968 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.029322 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.018999 13241 raft_consensus.cc:3055] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Stepping down as leader of term 4
I20260807 19:50:09.030882 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.031985 12927 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:08.950661 13325 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:09.034853 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.034291 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.043028 9021 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:09.046954 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.047215 8856 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.048305 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.051934 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.052819 12846 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:50:09.053226 12846 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.032089 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.053331 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.056557 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.042150 12775 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.058151 13325 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.059019 12775 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:09.059749 12846 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:09.039238 12927 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.036707 13241 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:09.066839 8968 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.009087 13101 consensus_queue.cc:1059] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260807 19:50:09.071442 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 168913us
I20260807 19:50:09.080318 9019 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
I20260807 19:50:09.066147 12927 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:50:09.078222 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 164653us
I20260807 19:50:09.084612 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:09.085493 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.043728 12681 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:09.090372 9898 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:09.067077 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.045873 13280 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260807 19:50:09.068522 13325 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:09.099717 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.102470 8856 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:09.104969 8856 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:09.105648 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 60821.2us
I20260807 19:50:09.106776 8856 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:09.111012 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 79625us
I20260807 19:50:09.114321 13125 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:09.114825 13125 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:09.115181 13125 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:09.097913 13010 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:09.098198 13041 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:50:09.126575 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.128886 13361 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:09.129393 13361 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.129727 13361 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:09.131346 8856 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.132331 8856 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:09.108083 13229 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:09.134927 8968 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.137022 8849 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:09.139371 8968 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.133512 8856 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.111802 12681 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:09.133347 13038 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:09.147610 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.136312 13162 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:09.137517 13011 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:09.144392 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:09.160686 8968 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.141906 13216 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:09.170934 8776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:51024
I20260807 19:50:09.171690 8776 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:09.171732 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.172222 8776 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:09.173799 8779 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 4 to 6, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:50:09.165904 13350 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260807 19:50:09.176482 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.149691 12681 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.157193 13162 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.159425 13051 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:09.182124 13007 consensus_queue.cc:1059] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260807 19:50:09.182845 13239 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:09.138760 13041 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.185423 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.189374 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.185541 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.128942 13241 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.159184 13038 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:09.165679 13216 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.191359 8778 raft_consensus.cc:2410] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 19:50:09.178529 13101 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:09.194782 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.185477 13162 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:09.183502 12681 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.191656 8789 raft_consensus.cc:2393] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 3: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:09.197844 8790 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:09.206743 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.193347 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:09.199693 8783 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:09.202534 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.184588 13011 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:09.199236 13101 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:09.188063 13239 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:09.212774 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2023) took 1143 ms. Trace:
I20260807 19:50:09.197337 13216 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.250411 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.265136 9020 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260807 19:50:09.265798 9008 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:09.266796 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.231464 13318 raft_consensus.cc:3055] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:09.271440 13318 raft_consensus.cc:740] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:09.272505 13318 consensus_queue.cc:260] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.273769 13318 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:09.274013 13041 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.240319 13011 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:09.250140 13101 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:50:09.277509 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.250830 13239 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:09.243664 8787 rpcz_store.cc:276] 0807 19:50:07.952587 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:08.305717 (+353130us) service_pool.cc:224] Handling call
0807 19:50:08.306140 (+ 423us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:08.306613 (+ 473us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 19:50:08.804398 (+497785us) log.cc:844] Serialized 14 byte log entry
0807 19:50:09.095325 (+290927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:09.095387 (+ 62us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:09.095437 (+ 50us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:09.095505 (+ 68us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:09.095624 (+ 119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2030,"prepare.queue_time_us":318070,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":59,"raft.queue_time_us":2540,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":11746,"thread_start_us":506385,"threads_started":2,"wal-append.queue_time_us":365388}
I20260807 19:50:09.267516 9018 raft_consensus.cc:2376] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8. Re-sending same reply.
I20260807 19:50:09.227151 12846 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.229475 13323 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:09.235044 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.187896 13051 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:50:09.292019 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 98791us
W20260807 19:50:09.298069 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.275612 13011 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:09.316406 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.324262 8776 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.328773 8968 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:09.328794 8778 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:09.302399 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.326026 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.345983 8776 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.293540 13280 raft_consensus.cc:2749] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:09.353039 13202 raft_consensus.cc:2749] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260807 19:50:09.353575 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.354291 8972 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:50:09.354916 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 55834.2us
I20260807 19:50:09.343091 9885 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260807 19:50:09.314841 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:09.293962 13371 log.cc:927] Time spent T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.053s user 0.001s sys 0.000s
W20260807 19:50:09.360123 8740 connection.cc:256] RPC call timeout handler was delayed by -5.2444044998846948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22450000000000003s for a response
I20260807 19:50:09.370209 13101 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.343740 13321 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.377506 13101 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.323866 13350 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260807 19:50:09.301425 13051 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:09.356590 12943 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
W20260807 19:50:09.380491 8632 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38426 (request call id 209) took 14147 ms (client timeout 15000 ms). Trace:
I20260807 19:50:09.351322 13229 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:09.344879 13253 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:09.359354 13065 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 19:50:09.395612 8740 connection.cc:256] RPC call timeout handler was delayed by -4.2265969631262124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
I20260807 19:50:09.387866 13358 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:09.399194 8740 connection.cc:256] RPC call timeout handler was delayed by -4.2265969631262124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
W20260807 19:50:09.416678 8740 connection.cc:256] RPC call timeout handler was delayed by -4.1815969632007182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260807 19:50:09.382058 13041 consensus_queue.cc:1048] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260807 19:50:09.419148 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 293213us
I20260807 19:50:09.420688 8906 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
W20260807 19:50:09.417079 8740 connection.cc:256] RPC call timeout handler was delayed by -4.0150969631504267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260807 19:50:09.421799 8740 connection.cc:256] RPC call timeout handler was delayed by -4.0150969631504267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260807 19:50:09.422104 8740 connection.cc:256] RPC call timeout handler was delayed by -3.9079969630111009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
W20260807 19:50:09.422375 8740 connection.cc:256] RPC call timeout handler was delayed by -3.9079969630111009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
I20260807 19:50:09.422286 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:09.422885 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7846969631500542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
I20260807 19:50:09.422792 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.423471 8904 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:09.424520 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7846969631500542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2156s for a response
W20260807 19:50:09.424800 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7558969631791115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
I20260807 19:50:09.397117 13065 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:50:09.425025 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7558969631791115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
I20260807 19:50:09.423346 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.424166 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:09.428857 8740 connection.cc:256] RPC call timeout handler was delayed by -3.74329696316272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260807 19:50:09.424049 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.428259 8915 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:09.429209 8902 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:09.429226 8903 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:09.396409 13325 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:09.429735 8908 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
W20260807 19:50:09.429229 8740 connection.cc:256] RPC call timeout handler was delayed by -3.74329696316272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
W20260807 19:50:09.438448 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7297969630453736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260807 19:50:09.415517 12943 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:09.438570 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.399976 13253 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:09.439810 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7297969630453736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260807 19:50:09.445217 8908 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:09.464804 8906 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:09.442123 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.442399 8849 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:09.470367 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7216969630680978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
W20260807 19:50:09.470811 8740 connection.cc:256] RPC call timeout handler was delayed by -3.7216969630680978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22260000000000002s for a response
I20260807 19:50:09.469658 8903 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
W20260807 19:50:09.471117 8740 connection.cc:256] RPC call timeout handler was delayed by -3.0601969631388783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2961s for a response
I20260807 19:50:09.396847 13229 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.431470 13065 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.444633 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:09.471482 8740 connection.cc:256] RPC call timeout handler was delayed by -3.0601969631388783s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2961s for a response
I20260807 19:50:09.471891 8915 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
W20260807 19:50:09.473865 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.471306 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.477447 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.478519 8915 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:09.479492 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54662 (request call id 1239) took 6337 ms. Trace:
I20260807 19:50:09.479682 12846 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.479700 8901 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:09.485347 13253 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.442019 13382 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260807 19:50:09.477035 13325 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.489964 13125 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.491426 13229 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.491628 13325 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:09.493773 13216 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.494395 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.495446 9885 raft_consensus.cc:2410] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260807 19:50:09.495826 13216 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.496654 9019 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:09.488137 8740 leader_election.cc:400] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:09.499130 8740 leader_election.cc:403] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:50:09.501619 8904 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:09.477964 8906 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:09.492075 13125 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.439502 13010 raft_consensus.cc:697] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:09.479900 8910 rpcz_store.cc:276] 0807 19:50:03.137556 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:07.106272 (+3968716us) service_pool.cc:224] Handling call
0807 19:50:07.106863 (+ 591us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:50:08.231703 (+1124840us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:50:08.819885 (+588182us) log.cc:844] Serialized 14 byte log entry
0807 19:50:09.047691 (+227806us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:50:09.051138 (+ 3447us) log.cc:844] Serialized 52 byte log entry
0807 19:50:09.051292 (+ 154us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:09.054607 (+ 3315us) log.cc:844] Serialized 14 byte log entry
0807 19:50:09.055716 (+ 1109us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:09.071709 (+ 15993us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:09.221561 (+149852us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b1800329680. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784fde62 00007f6878500121 00007f68785000d9 00007f6878500069 00007f6878500031 00007f68784ff32d 00000000003aa554 00000000003aa399 00007f68784e11ea
0807 19:50:09.475153 (+253592us) raft_consensus.cc:1685] finished
0807 19:50:09.475181 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:09.475349 (+ 168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1132013,"prepare.queue_time_us":405545,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":73,"raft.queue_time_us":1181201,"raft.run_cpu_time_us":479,"raft.run_wall_time_us":11851,"spinlock_wait_cycles":22929664,"thread_start_us":744381,"threads_started":2,"wal-append.queue_time_us":265102}
W20260807 19:50:09.505511 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 63230us
I20260807 19:50:09.505631 8901 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:09.507674 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.509876 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.510900 13253 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260807 19:50:09.511003 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.512224 8908 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:50:09.490010 12927 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.514446 12927 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.515581 13383 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:50:09.516201 13383 raft_consensus.cc:697] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:09.517001 13162 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.430713 13358 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.510416 8906 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
W20260807 19:50:09.523492 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 194403us
I20260807 19:50:09.443094 13011 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.538049 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.538796 8779 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:09.524122 8902 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:09.442864 8769 tablet_service.cc:1558] Processing DeleteTablet for tablet 300f918f185249e6895ab57edd060d31 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 7)) from {username='slave'} at 127.0.0.1:60644
I20260807 19:50:09.546420 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:09.529017 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.539588 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.540246 12609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52256 (request call id 68) took 8675 ms. Trace:
W20260807 19:50:09.528651 8740 raft_consensus.cc:2686] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260807 19:50:09.390164 8632 rpcz_store.cc:269] 0807 19:49:55.232637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:49:55.232910 (+ 273us) service_pool.cc:224] Handling call
0807 19:50:09.380230 (+14147320us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:50:02.240403 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:50:02.423796 (+183393us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:50:02.445699 (+ 21903us) write_op.cc:435] Acquired schema lock
0807 19:50:02.449522 (+ 3823us) tablet.cc:662] Decoding operations
0807 19:50:02.562503 (+112981us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:50:02.576636 (+ 14133us) write_op.cc:641] Partition lock acquired for write op
0807 19:50:02.578426 (+ 1790us) tablet.cc:685] Acquiring locks for 2 operations
0807 19:50:02.596955 (+ 18529us) tablet.cc:701] Row locks acquired
0807 19:50:02.596981 (+ 26us) write_op.cc:260] PREPARE: finished
0807 19:50:02.597144 (+ 163us) write_op.cc:270] Start()
0807 19:50:02.597257 (+ 113us) write_op.cc:276] Timestamp: P: 1786132202597094 usec, L: 0
0807 19:50:02.597286 (+ 29us) op_driver.cc:348] REPLICATION: starting
0807 19:50:02.598117 (+ 831us) log.cc:844] Serialized 923 byte log entry
0807 19:50:02.867655 (+269538us) op_driver.cc:464] REPLICATION: finished
0807 19:50:03.041816 (+174161us) write_op.cc:301] APPLY: starting
0807 19:50:03.041935 (+ 119us) tablet.cc:1367] starting BulkCheckPresence
0807 19:50:03.042054 (+ 119us) tablet.cc:1370] finished BulkCheckPresence
0807 19:50:03.042170 (+ 116us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:50:03.042540 (+ 370us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:50:03.042758 (+ 218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 19:50:03.042802 (+ 44us) write_op.cc:312] APPLY: finished
0807 19:50:03.043020 (+ 218us) log.cc:844] Serialized 29 byte log entry
0807 19:50:03.043238 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:03.043331 (+ 93us) write_op.cc:454] Released schema lock
0807 19:50:03.043370 (+ 39us) write_op.cc:341] FINISH: Updating metrics
0807 19:50:03.043788 (+ 418us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:03.043816 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":40823,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":137481,"apply.run_cpu_time_us":1932,"apply.run_wall_time_us":1932,"num_ops":2,"prepare.queue_time_us":57943,"prepare.run_cpu_time_us":4412,"prepare.run_wall_time_us":358075,"replication_time_us":255210,"thread_start_us":203653,"threads_started":2}]]}
I20260807 19:50:09.544828 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.543308 13162 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:09.547706 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 59707.9us
I20260807 19:50:09.547483 8775 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:09.548313 8776 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:09.549866 8915 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:50:09.554847 13010 consensus_queue.cc:237] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.556989 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.558879 8783 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.563596 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.564280 8787 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:09.566255 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:09.567936 8740 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.568933 8740 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.568737 12943 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.571043 13011 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.572269 13361 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.575865 13361 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.573184 13327 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:09.517187 13383 consensus_queue.cc:237] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.576737 13327 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:09.577760 13327 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.573650 13325 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:50:09.563761 13358 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:09.586223 13102 consensus_meta.cc:287] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.681s user 0.015s sys 0.008s
I20260807 19:50:09.590332 9023 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:09.592221 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.593200 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.593451 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.593892 8902 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:09.594341 13244 raft_consensus.cc:3055] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:09.594791 13244 raft_consensus.cc:740] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:09.595929 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.597425 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.598608 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.599663 13237 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:09.600656 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.601233 13237 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:09.601928 13237 consensus_queue.cc:1243] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer 671d8754170646eaac5748745eba58b4 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260807 19:50:09.602154 13241 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:09.602517 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.603713 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.604480 8901 raft_consensus.cc:2393] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:09.622699 8776 raft_consensus.cc:2468] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:09.627535 13344 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:09.627986 13344 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:09.629004 13344 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.630340 13344 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.632513 8906 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:09.632902 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.633560 8910 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
W20260807 19:50:09.634790 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 64972.6us
I20260807 19:50:09.622200 13325 raft_consensus.cc:697] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:09.637743 13325 consensus_queue.cc:237] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.628074 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.644641 8787 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:09.648202 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.652611 8779 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
W20260807 19:50:09.697544 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.730190 9023 raft_consensus.cc:2368] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10.
I20260807 19:50:09.735936 9027 raft_consensus.cc:1275] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260807 19:50:09.735841 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.736836 9018 raft_consensus.cc:2376] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9. Re-sending same reply.
I20260807 19:50:09.631966 13051 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.738355 9021 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:09.589408 12681 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:09.739197 8783 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:09.738802 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.739760 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.738773 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.645504 13237 raft_consensus.cc:2764] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260807 19:50:09.742818 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.728627 13358 raft_consensus.cc:2764] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260807 19:50:09.735351 13038 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
W20260807 19:50:09.752940 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.755554 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:09.757005 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:09.756289 8901 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:09.624873 13239 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.745450 13274 raft_consensus.cc:2764] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260807 19:50:09.645754 12775 consensus_queue.cc:1059] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260807 19:50:09.759791 8911 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:09.767472 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.769862 8775 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
W20260807 19:50:09.768774 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 170223us
I20260807 19:50:09.772970 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.773684 8910 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:09.778319 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.778369 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.780941 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.781354 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.781771 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.789080 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.779037 9019 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:09.790373 9008 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:09.776679 9892 raft_consensus.cc:2955] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 4 to 6, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:50:09.781840 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.794176 9023 raft_consensus.cc:2368] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:09.800478 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.804999 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:09.807834 13390 log.cc:927] Time spent T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.186s user 0.000s sys 0.000s
I20260807 19:50:09.821610 8783 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260807 19:50:09.822542 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 44056.6us
I20260807 19:50:09.824031 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:09.824710 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.548448 12609 rpcz_store.cc:276] 0807 19:50:00.864985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:00.865130 (+ 145us) service_pool.cc:224] Handling call
0807 19:50:01.473560 (+608430us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b2c0007bcd8. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6879effc71 00007f6879f0ec6a 00007f6879f0e8f3 00007f6879f0e891 00007f687b63f7d1 00007f687b578dde 00007f687b5ffb29 00007f687b600b4a 00007f687b5ffcfb 00007f687b60b41c 00007f687b60aa93
0807 19:50:01.475477 (+ 1917us) log.cc:844] Serialized 494 byte log entry
0807 19:50:01.863285 (+387808us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b180032a5e0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784fde62 00007f6878500121 00007f68785000d9 00007f6878500069 00007f6878500031 00007f68784ff32d 00000000003aa554 00000000003aa399 00007f68784e11ea
0807 19:50:09.540101 (+7676816us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33979,"raft.queue_time_us":50751,"raft.run_cpu_time_us":4442,"raft.run_wall_time_us":98637,"spinlock_wait_cycles":16129152,"thread_start_us":251379,"threads_started":1,"wal-append.queue_time_us":199538,"wal-append.run_cpu_time_us":3127,"wal-append.run_wall_time_us":1210397}
I20260807 19:50:09.825551 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:09.825914 12775 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:09.828534 12775 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:09.828929 12775 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:09.814512 13239 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.792960 13327 raft_consensus.cc:3055] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:09.817574 13051 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:09.819025 12681 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:09.868168 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.763230 13038 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.837052 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.827286 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:09.822042 8783 pending_rounds.cc:85] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:50:09.877360 8783 pending_rounds.cc:107] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260807 19:50:09.877656 8783 raft_consensus.cc:2889] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:09.823271 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.830755 13065 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.859349 9017 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:09.862243 9022 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:09.864641 9021 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:09.869372 9019 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:09.884208 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:09.884903 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 61707.4us
I20260807 19:50:09.884917 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.885996 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.886786 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.889443 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.892146 13065 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:09.898422 8613 catalog_manager.cc:5028] TS 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): tablet 300f918f185249e6895ab57edd060d31 (table test-workload [id=f3fe82543e9d4862a0ebfb272d9749de]) successfully deleted
I20260807 19:50:09.900321 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.866941 13327 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:09.901646 9017 raft_consensus.cc:2393] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 10: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:50:09.901921 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 70170.8us
I20260807 19:50:09.902225 8775 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 4. Current term is 5. Ops: []
I20260807 19:50:09.903697 8972 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:09.904309 9023 raft_consensus.cc:1240] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:09.877019 13011 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:09.913451 9022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "95f757dd3f63420fad091c3802390158"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:09.914397 9022 raft_consensus.cc:606] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:09.914350 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.915216 9019 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:09.917066 8856 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:09.917227 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:09.919281 8856 leader_election.cc:400] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10.
I20260807 19:50:09.919595 8856 leader_election.cc:403] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:50:09.920077 8783 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:09.920100 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.925347 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.926107 8901 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.925643 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.927834 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.926800 8906 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:09.939276 9017 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.937873 8740 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.940261 8783 raft_consensus.cc:1275] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:50:09.943200 8616 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a898110875ba40ab8e512e124aff4fb2 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260807 19:50:09.943632 13327 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.945176 13327 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:09.944082 13404 raft_consensus.cc:2764] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260807 19:50:09.950953 13038 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.951138 13321 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:09.952492 12775 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.953364 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.925486 13011 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:09.956856 13011 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:09.959244 13403 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.333s user 0.001s sys 0.000s
I20260807 19:50:09.963601 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 73) took 7264 ms. Trace:
I20260807 19:50:09.954699 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:09.977982 9023 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
W20260807 19:50:09.980819 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 59579.2us
I20260807 19:50:09.968000 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.985096 8783 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:09.927398 13325 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:09.981791 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.989003 8901 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:09.987598 9019 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 10.
I20260807 19:50:09.990085 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:09.992029 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:09.992657 9008 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:09.926849 13383 raft_consensus.cc:2764] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260807 19:50:09.989215 13325 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:09.988390 13358 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:09.995453 13325 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:09.997335 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:09.997563 13344 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:09.998059 13344 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:09.999518 8787 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.954612 12775 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:10.000674 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 66061.1us
W20260807 19:50:10.001578 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 98398.3us
I20260807 19:50:09.998462 13344 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:09.965294 13384 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:09.938354 13065 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260807 19:50:10.012128 13325 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.034595 13325 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:10.037302 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.022814 8787 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:10.040247 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:10.041074 8779 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:10.005836 8783 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:09.995428 13321 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260807 19:50:10.045030 13318 raft_consensus.cc:3055] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:10.015069 9008 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
I20260807 19:50:10.045454 13318 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:10.015920 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.046911 13207 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:10.046502 13318 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:10.047677 8972 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.048027 13318 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:10.048466 13207 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:10.049492 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.050143 13207 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:10.052171 8972 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:10.017102 9017 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
I20260807 19:50:09.964061 8629 rpcz_store.cc:276] 0807 19:50:02.698948 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:02.699115 (+ 167us) service_pool.cc:224] Handling call
0807 19:50:02.870671 (+171556us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b5800001420. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6ff0 00007f68813f1d36 00007f68814fa4be 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc 00007f687f9b2592 00007f687993e07c 00007f687993d4b6
0807 19:50:09.963501 (+7092830us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:50:09.082408 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:50:09.132856 (+ 50448us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:50:09.132910 (+ 54us) write_op.cc:435] Acquired schema lock
0807 19:50:09.132945 (+ 35us) tablet.cc:662] Decoding operations
0807 19:50:09.151830 (+ 18885us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:50:09.164789 (+ 12959us) write_op.cc:641] Partition lock acquired for write op
0807 19:50:09.164840 (+ 51us) tablet.cc:685] Acquiring locks for 2 operations
0807 19:50:09.203195 (+ 38355us) tablet.cc:701] Row locks acquired
0807 19:50:09.203228 (+ 33us) write_op.cc:260] PREPARE: finished
0807 19:50:09.203450 (+ 222us) write_op.cc:270] Start()
0807 19:50:09.203610 (+ 160us) write_op.cc:276] Timestamp: P: 1786132209203366 usec, L: 0
0807 19:50:09.203645 (+ 35us) op_driver.cc:348] REPLICATION: starting
0807 19:50:09.218316 (+ 14671us) log.cc:844] Serialized 913 byte log entry
0807 19:50:09.721854 (+503538us) op_driver.cc:464] REPLICATION: finished
0807 19:50:09.907633 (+185779us) write_op.cc:301] APPLY: starting
0807 19:50:09.907738 (+ 105us) tablet.cc:1367] starting BulkCheckPresence
0807 19:50:09.907911 (+ 173us) tablet.cc:1370] finished BulkCheckPresence
0807 19:50:09.908059 (+ 148us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:50:09.908745 (+ 686us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:50:09.909090 (+ 345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0807 19:50:09.909161 (+ 71us) write_op.cc:312] APPLY: finished
0807 19:50:09.909453 (+ 292us) log.cc:844] Serialized 29 byte log entry
0807 19:50:09.909814 (+ 361us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:09.909909 (+ 95us) write_op.cc:454] Released schema lock
0807 19:50:09.909968 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0807 19:50:09.910526 (+ 558us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:09.910554 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":33877,"reported_tablets":2,"spinlock_wait_cycles":3532416,"child_traces":[["op",{"apply.queue_time_us":185379,"apply.run_cpu_time_us":2685,"apply.run_wall_time_us":2918,"num_ops":2,"prepare.queue_time_us":298824,"prepare.run_cpu_time_us":29117,"prepare.run_wall_time_us":348928,"raft.queue_time_us":234848,"raft.run_cpu_time_us":35746,"raft.run_wall_time_us":140002,"replication_time_us":518025,"thread_start_us":617239,"threads_started":4,"wal-append.queue_time_us":266191}]]}
I20260807 19:50:10.019198 8968 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:10.002712 13358 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:10.024710 8901 raft_consensus.cc:1275] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:10.017890 8740 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:10.054486 9021 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260807 19:50:10.026260 8856 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.025970 13244 consensus_queue.cc:260] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.026938 12775 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:10.027169 13385 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:10.026865 13038 consensus_queue.cc:1048] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260807 19:50:10.002345 13404 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260807 19:50:10.057387 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 65720.6us
W20260807 19:50:10.052793 13410 log.cc:927] Time spent T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260807 19:50:10.060201 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.057196 12775 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:10.066030 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46935.6us
I20260807 19:50:10.067368 12775 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:10.061118 13325 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:10.069103 8779 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:10.072675 8972 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:10.077246 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.058919 13382 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:10.075981 13241 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:10.071751 13325 raft_consensus.cc:697] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:50:10.078711 13409 log.cc:927] Time spent T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.113s user 0.001s sys 0.000s
I20260807 19:50:10.073118 8740 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:10.059254 13404 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:10.082129 8856 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.070489 13059 raft_consensus.cc:2764] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260807 19:50:10.087815 12681 raft_consensus.cc:3055] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:10.071791 13051 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:10.083283 13086 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:10.085666 13241 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:10.092973 8972 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.072563 13244 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:10.098079 12681 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:10.117084 12681 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.118355 12681 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:10.100847 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.088145 13325 consensus_queue.cc:237] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.108937 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:10.098397 13361 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:10.104696 13041 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:10.104486 13280 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:10.110795 13207 raft_consensus.cc:2749] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:10.095779 13404 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:10.097769 13382 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:10.096048 13385 consensus_queue.cc:1059] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260807 19:50:10.113864 13318 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 19:50:10.093667 13162 consensus_queue.cc:1059] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260807 19:50:10.104112 13241 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:10.134446 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:50:10.113981 13413 log.cc:927] Time spent T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.168s user 0.002s sys 0.000s
I20260807 19:50:10.133499 13011 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:10.147110 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46378.4us
I20260807 19:50:10.080090 13358 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.128682 12775 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:10.160677 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:10.212571 8901 raft_consensus.cc:3060] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 9
I20260807 19:50:10.179672 8906 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
I20260807 19:50:10.158154 13059 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:10.142925 13237 raft_consensus.cc:2764] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260807 19:50:10.147145 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:10.178320 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 43945.5us
I20260807 19:50:10.224920 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:10.191849 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:10.241495 8740 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:10.261405 8968 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:10.272003 8968 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.242220 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:10.273921 9008 raft_consensus.cc:606] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:10.279292 8911 raft_consensus.cc:2368] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 9.
I20260807 19:50:10.170116 13280 raft_consensus.cc:697] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.187196 13342 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:10.218679 13071 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260807 19:50:10.306362 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.240196 13359 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:10.262615 12775 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:50:10.315074 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 89188.3us
I20260807 19:50:10.324671 13382 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.328951 9892 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:10.291059 13384 raft_consensus.cc:2749] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10.
I20260807 19:50:10.336140 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:10.271190 13059 consensus_queue.cc:1059] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260807 19:50:10.310415 13216 raft_consensus.cc:2804] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:10.269914 13011 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.312284 13342 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:10.314916 13359 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:10.331734 12943 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:10.360716 8911 raft_consensus.cc:1275] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260807 19:50:10.330427 13327 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:10.365597 8901 raft_consensus.cc:2468] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:10.383412 8903 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:10.368865 13342 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:10.396809 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 124900us
I20260807 19:50:10.382040 13359 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:50:10.398902 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.402302 8788 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:10.387089 13216 raft_consensus.cc:697] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.409651 8968 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.368633 13361 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 10 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:10.383508 13274 consensus_queue.cc:1059] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260807 19:50:10.415498 13051 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader election won for term 11
I20260807 19:50:10.416137 13051 raft_consensus.cc:697] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.416060 13361 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.417491 13051 consensus_queue.cc:237] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:10.419682 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.420828 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.421145 8968 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.395865 12943 raft_consensus.cc:697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.431114 13394 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:10.428231 12609 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:10.432912 12609 raft_consensus.cc:697] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.433892 12609 consensus_queue.cc:237] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.435593 8968 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.436501 13394 raft_consensus.cc:2804] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:10.437041 13394 raft_consensus.cc:697] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.437990 13394 consensus_queue.cc:237] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.441931 8968 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:10.443185 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:10.431715 13102 consensus_queue.cc:1048] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
W20260807 19:50:10.448699 13438 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.113s user 0.001s sys 0.000s
I20260807 19:50:10.521763 13183 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:10.449698 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.522452 13183 raft_consensus.cc:697] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.523412 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.411229 13327 raft_consensus.cc:697] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:10.416185 13274 raft_consensus.cc:3055] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:10.524078 13274 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:10.523666 13183 consensus_queue.cc:237] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:10.524554 13327 consensus_queue.cc:237] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.525066 13274 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:10.526351 13274 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:10.527036 13241 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:10.527714 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 84607.5us
I20260807 19:50:10.455181 9018 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
W20260807 19:50:10.456468 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 133654us
I20260807 19:50:10.464531 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.530301 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:10.531473 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.449210 13011 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:10.461966 13244 raft_consensus.cc:993] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:10.532758 13244 raft_consensus.cc:1081] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Signalling peer 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:10.462591 13393 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:10.534066 13393 raft_consensus.cc:697] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:10.534998 13393 consensus_queue.cc:237] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.502215 8790 raft_consensus.cc:1275] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260807 19:50:10.505089 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.505429 13429 raft_consensus.cc:3055] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:10.405457 13280 consensus_queue.cc:237] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.426013 12775 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:10.510862 13434 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
W20260807 19:50:10.511962 13428 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.254s user 0.001s sys 0.000s
W20260807 19:50:10.512149 13411 log.cc:927] Time spent T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.560s user 0.004s sys 0.000s
I20260807 19:50:10.514460 13383 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:10.516541 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:10.518963 13207 raft_consensus.cc:993] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
W20260807 19:50:10.519218 13427 log.cc:927] Time spent T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.188s user 0.001s sys 0.000s
I20260807 19:50:10.520536 8913 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:10.533439 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.578127 8787 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:10.545842 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.549469 8856 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:10.551491 9850 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:10.552443 8901 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260807 19:50:10.573665 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.572152 8972 leader_election.cc:400] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 9.
I20260807 19:50:10.580866 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.581627 8776 raft_consensus.cc:2368] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:10.581000 8972 leader_election.cc:403] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260807 19:50:10.541119 13429 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:10.583241 13318 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader election won for term 8
W20260807 19:50:10.586899 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.594812 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:10.595582 8790 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:10.597404 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.597620 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.598676 8776 raft_consensus.cc:2393] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:10.584239 13244 raft_consensus.cc:1081] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Signalling peer 8e9ad2b5e2a94d4faff90af735c6d792 to start an election
I20260807 19:50:10.611827 8787 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
W20260807 19:50:10.619587 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 256505us
I20260807 19:50:10.517376 12943 consensus_queue.cc:237] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.626395 13012 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260807 19:50:10.627521 13012 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:10.629027 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.629029 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.624636 9015 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:10.582693 13383 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:10.656947 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.628010 13012 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:10.662158 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.663969 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.630681 13241 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:10.668063 8787 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:50:10.672979 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 85628.2us
I20260807 19:50:10.581447 13216 consensus_queue.cc:237] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.660219 13456 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:10.712985 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.713896 8787 raft_consensus.cc:2393] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 10: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:10.714566 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.715456 8776 raft_consensus.cc:2410] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260807 19:50:10.722548 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.712642 13318 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:10.843098 8908 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1693) took 1027 ms. Trace:
W20260807 19:50:10.839517 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.853613 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.843557 8908 rpcz_store.cc:276] 0807 19:50:09.815333 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:09.819959 (+ 4626us) service_pool.cc:224] Handling call
0807 19:50:09.832030 (+ 12071us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:09.832439 (+ 409us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 19:50:09.938272 (+105833us) log.cc:844] Serialized 14 byte log entry
0807 19:50:10.186073 (+247801us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:10.207511 (+ 21438us) log.cc:844] Serialized 26 byte log entry
0807 19:50:10.269629 (+ 62118us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:10.269672 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:10.269742 (+ 70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:10.842828 (+573086us) raft_consensus.cc:1685] finished
0807 19:50:10.842853 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:10.843031 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131441,"prepare.queue_time_us":78287,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":13129,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":171,"spinlock_wait_cycles":34048,"thread_start_us":199003,"threads_started":3,"wal-append.queue_time_us":174433}
W20260807 19:50:10.892257 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51714.1us
I20260807 19:50:10.893193 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:10.910501 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.916883 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:10.841821 13384 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:10.917796 8776 raft_consensus.cc:2468] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
W20260807 19:50:10.922183 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 271225us
W20260807 19:50:10.928514 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.935058 13071 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:10.936406 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:10.668192 13429 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.943778 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:10.921890 13435 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.355s user 0.002s sys 0.000s
I20260807 19:50:10.946607 8787 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:10.952791 13041 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:10.957314 13318 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:10.957727 13041 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:10.962006 13461 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.347s user 0.000s sys 0.001s
I20260807 19:50:10.903764 13459 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:10.960069 13041 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:10.784845 13456 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.935796 13071 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.928514 13384 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:10.938897 13466 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659) with higher term. Message: Invalid argument: T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 9.
W20260807 19:50:11.023425 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 451354us
W20260807 19:50:11.095024 13477 log.cc:927] Time spent T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.184s user 0.000s sys 0.000s
I20260807 19:50:11.127055 13480 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:11.127664 13125 raft_consensus.cc:3055] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:11.128301 13125 raft_consensus.cc:740] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:11.129779 13488 log.cc:927] Time spent T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.171s user 0.001s sys 0.000s
I20260807 19:50:11.129343 13125 consensus_queue.cc:260] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.131480 13444 raft_consensus.cc:3055] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:11.131933 13444 raft_consensus.cc:740] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:11.134608 13071 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:11.135772 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.136443 8775 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Handling vote request from an unknown peer 671d8754170646eaac5748745eba58b4
I20260807 19:50:11.137006 8775 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 19:50:11.137672 8778 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260807 19:50:11.138804 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.137957 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.101354 12609 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.140025 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.154042 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.107488 13429 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:50:11.155093 8790 consensus_meta.cc:287] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.535s user 0.010s sys 0.000s
I20260807 19:50:11.157481 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.158819 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.162081 8778 raft_consensus.cc:2368] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10.
I20260807 19:50:11.164790 8790 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:50:11.168752 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:11.173712 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.127781 13480 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.175274 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.303004 13456 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:11.303507 13274 consensus_meta.cc:287] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.777s user 0.009s sys 0.002s
I20260807 19:50:11.299578 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.304749 8776 raft_consensus.cc:2468] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:11.303684 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.304423 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.202625 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.277762 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:11.307437 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 151589us
I20260807 19:50:11.269148 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:11.261296 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:11.314370 8778 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:11.313519 8908 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
W20260807 19:50:11.161562 13404 consensus_meta.cc:287] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.974s user 0.005s sys 0.007s
I20260807 19:50:11.317765 8901 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:11.161842 13344 consensus_meta.cc:287] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 1.135s user 0.013s sys 0.000s
W20260807 19:50:11.291204 13342 consensus_meta.cc:287] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.876s user 0.013s sys 0.000s
W20260807 19:50:11.205926 12681 consensus_meta.cc:287] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.087s user 0.011s sys 0.004s
I20260807 19:50:11.320688 8903 raft_consensus.cc:2393] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 7: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
W20260807 19:50:11.311269 13359 consensus_meta.cc:287] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.882s user 0.012s sys 0.000s
I20260807 19:50:11.329542 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.330371 8790 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:11.295228 13492 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:11.255929 13493 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.334414 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.335057 8776 raft_consensus.cc:2468] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:11.339517 8901 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:11.173854 13321 raft_consensus.cc:3055] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:11.331316 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:11.185616 8789 raft_consensus.cc:2368] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 10. Current term is 11.
I20260807 19:50:11.346508 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:11.282858 13012 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:11.298110 13384 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.344208 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.256515 13466 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:11.362699 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:11.363365 8775 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:11.351956 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.282755 13383 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.298592 13323 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260807 19:50:11.353703 9020 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:50:11.358419 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.359464 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.356840 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.372053 9008 raft_consensus.cc:2393] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:11.360653 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.365623 8789 raft_consensus.cc:2410] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:50:11.366577 8901 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [term 10 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260807 19:50:11.371272 9023 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:11.373869 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.374632 9022 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
I20260807 19:50:11.375775 9021 raft_consensus.cc:1275] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:50:11.349653 13321 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:11.383042 9027 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:11.393520 8904 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:11.350016 13183 consensus_queue.cc:1048] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260807 19:50:11.335899 13342 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.331754 13344 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.324801 13404 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.333571 13359 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.162555 12609 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:11.417452 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:11.412765 8775 raft_consensus.cc:2410] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260807 19:50:11.428947 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 152363us
I20260807 19:50:11.353499 13493 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.406864 13274 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:11.455531 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.456243 8901 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:50:11.418318 13383 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.357518 13492 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.460253 8788 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:11.418460 13459 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:11.459326 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:50:11.371316 13466 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.453835 13474 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.484383 9022 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260807 19:50:11.488042 8776 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:50:11.493711 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 137684us
I20260807 19:50:11.489059 13485 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:11.495774 13485 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:11.496225 13485 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:11.497105 13502 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.078s user 0.000s sys 0.001s
W20260807 19:50:11.498944 13464 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.538s user 0.000s sys 0.010s
I20260807 19:50:11.500422 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.501120 9008 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 3.
I20260807 19:50:11.502280 9008 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:11.479059 13383 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.504774 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.506577 13383 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.507109 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:11.508810 13466 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:11.462518 13274 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.511734 12609 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:11.512276 13444 consensus_queue.cc:260] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.514376 13444 raft_consensus.cc:3060] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:11.507112 13459 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:11.510058 13476 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.565s user 0.005s sys 0.004s
I20260807 19:50:11.489596 13474 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.527587 13474 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:11.490422 13321 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.530150 13321 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:11.531100 8630 catalog_manager.cc:5697] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 5 to 8, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 8 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:11.534202 9023 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:11.535125 13485 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.535512 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:11.537097 13474 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.537533 9015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 1242) took 1773 ms. Trace:
I20260807 19:50:11.537672 13474 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.533859 8630 catalog_manager.cc:5697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 5 to 7, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 7 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.540017 13474 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:11.540601 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:11.537931 9015 rpcz_store.cc:276] 0807 19:50:09.764294 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:09.771554 (+ 7260us) service_pool.cc:224] Handling call
0807 19:50:10.624431 (+852877us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:10.636443 (+ 12012us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:11.537311 (+900868us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54003ff2d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:11.537440 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892503424}
I20260807 19:50:11.543923 8630 catalog_manager.cc:5697] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 6 to 7, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 7 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:50:11.548149 13492 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:11.496428 13472 log.cc:927] Time spent T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.128s user 0.001s sys 0.000s
W20260807 19:50:11.548924 13479 log.cc:927] Time spent T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.591s user 0.009s sys 0.000s
I20260807 19:50:11.549634 13485 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:11.550771 13482 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:11.551419 13482 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.553782 13482 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.555105 9021 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
W20260807 19:50:11.558640 13462 log.cc:927] Time spent T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.667s user 0.003s sys 0.008s
I20260807 19:50:11.561113 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:11.561528 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 196) took 1230 ms. Trace:
I20260807 19:50:11.561936 8906 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:11.562460 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.561977 8779 rpcz_store.cc:276] 0807 19:50:10.341976 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:10.347716 (+ 5740us) service_pool.cc:224] Handling call
0807 19:50:10.383821 (+ 36105us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:10.395107 (+ 11286us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:10.395142 (+ 35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:10.404581 (+ 9439us) log.cc:844] Serialized 26 byte log entry
0807 19:50:10.599214 (+194633us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:50:10.848270 (+249056us) log.cc:844] Serialized 14 byte log entry
0807 19:50:10.916782 (+ 68512us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:10.916887 (+ 105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:11.558284 (+641397us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b240014ac98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f6878384594 00007f687838269b 00007f687853e6d9 00007f687853e365 00007f687853d9ad 00007f687b1872fe 00007f687b19ea7c 00007f687b19dda6 00007f687b19d9c8 00007f687b1a7411
0807 19:50:11.561209 (+ 2925us) raft_consensus.cc:1685] finished
0807 19:50:11.561312 (+ 103us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:11.561443 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":102779,"prepare.run_cpu_time_us":9602,"prepare.run_wall_time_us":83843,"raft.queue_time_us":218444,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":57654,"spinlock_wait_cycles":7899008,"thread_start_us":473999,"threads_started":3,"wal-append.queue_time_us":246301}
I20260807 19:50:11.567565 13359 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.568540 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.570246 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:11.571604 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.561560 8780 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
W20260807 19:50:11.572221 9018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1244) took 1807 ms (client timeout 1556 ms). Trace:
I20260807 19:50:11.575169 9023 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 10: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:11.538259 8778 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:11.576809 8789 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
W20260807 19:50:11.574008 9018 rpcz_store.cc:269] 0807 19:50:09.765011 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:09.765130 (+ 119us) service_pool.cc:224] Handling call
0807 19:50:11.572013 (+1806883us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400380754. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:11.572119 (+ 106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414424320}
I20260807 19:50:11.578414 9018 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:11.582723 8775 raft_consensus.cc:1240] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:11.585654 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.587643 9018 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260807 19:50:11.586537 8775 raft_consensus.cc:1275] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260807 19:50:11.588137 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:11.588641 9023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:55416
W20260807 19:50:11.589345 8737 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s user 0.006s sys 0.026s
I20260807 19:50:11.589349 9023 raft_consensus.cc:606] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Received request to transfer leadership to TS 671d8754170646eaac5748745eba58b4
W20260807 19:50:11.590081 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:11.590654 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:11.589593 13490 log.cc:927] Time spent T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.629s user 0.005s sys 0.000s
I20260807 19:50:11.601236 8775 pending_rounds.cc:85] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:50:11.610550 8906 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
I20260807 19:50:11.601624 8775 pending_rounds.cc:107] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260807 19:50:11.611595 8775 raft_consensus.cc:2889] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:11.546010 8630 catalog_manager.cc:5697] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
I20260807 19:50:11.613751 8630 catalog_manager.cc:5697] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 6 to 8, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 8 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.605216 8778 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:11.611832 9850 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4; no voters:
I20260807 19:50:11.616036 8630 catalog_manager.cc:5697] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.645663 8630 catalog_manager.cc:5697] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 4 to 6, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 6 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } }
W20260807 19:50:11.648130 8849 leader_election.cc:336] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.601s (SENT)
I20260807 19:50:11.649727 8849 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:11.650776 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.599520 13321 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:11.645035 13480 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.654554 13350 raft_consensus.cc:2804] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:11.652210 8775 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "95f757dd3f63420fad091c3802390158"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:51024
I20260807 19:50:11.657318 8778 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:11.657588 8775 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:11.658031 8775 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:11.610361 13012 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.648211 8630 catalog_manager.cc:5697] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 4 to 5, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.661306 8630 catalog_manager.cc:5697] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 5 to 6, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.664078 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:11.667635 8778 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:11.668252 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.668951 8903 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:11.664777 8906 raft_consensus.cc:3060] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:11.654912 13504 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:11.677332 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:11.686538 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 299806us
I20260807 19:50:11.687368 8903 raft_consensus.cc:2410] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:50:11.664266 8630 catalog_manager.cc:5697] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 4 to 5, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.684660 8775 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.689604 8630 catalog_manager.cc:5697] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 2 to 4, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.698573 8630 catalog_manager.cc:5697] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.702003 8630 catalog_manager.cc:5697] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.707381 8630 catalog_manager.cc:5697] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 reported cstate change: term changed from 3 to 4, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 4 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:11.726016 8906 raft_consensus.cc:2468] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:11.736454 13359 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.661957 13449 raft_consensus.cc:2749] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260807 19:50:11.760607 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 98807.1us
W20260807 19:50:11.761708 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:11.666833 13350 raft_consensus.cc:697] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:11.660245 13323 consensus_queue.cc:1048] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260807 19:50:11.782595 8849 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:11.776577 8775 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.782335 13350 consensus_queue.cc:237] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.793257 8849 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:11.762243 13342 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.777279 8906 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:11.762857 13504 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.800062 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.800751 8906 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:11.785019 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2092) took 1212 ms. Trace:
I20260807 19:50:11.787272 8903 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260807 19:50:11.662182 13012 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:11.789892 8778 raft_consensus.cc:1240] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:11.801666 8790 rpcz_store.cc:276] 0807 19:50:10.572926 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.366304 (+793378us) service_pool.cc:224] Handling call
0807 19:50:11.366639 (+ 335us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:11.368765 (+ 2126us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:11.368799 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:11.384111 (+ 15312us) log.cc:844] Serialized 26 byte log entry
0807 19:50:11.655233 (+271122us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:50:11.655273 (+ 40us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:11.655349 (+ 76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:11.784778 (+129429us) raft_consensus.cc:1685] finished
0807 19:50:11.784801 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:11.784956 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1520,"raft.queue_time_us":1644,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":15310,"thread_start_us":233768,"threads_started":1,"wal-append.queue_time_us":354636}
I20260807 19:50:11.810102 8790 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff"
dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:51024
I20260807 19:50:11.810683 8790 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:11.811056 8790 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:11.787652 8775 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:50:11.803483 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.784806 13359 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.814038 8903 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:11.803174 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:11.830103 8849 leader_election.cc:336] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.731s (SENT)
W20260807 19:50:11.831698 8849 leader_election.cc:336] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.002s (SENT)
I20260807 19:50:11.832137 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:11.814420 13361 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:11.833755 9886 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 19:50:11.834921 13012 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:11.835245 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:11.835902 13342 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:11.836484 13342 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:11.837788 13530 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260807 19:50:11.838536 13342 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.839151 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:11.840319 13342 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:50:11.841171 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:11.842557 8778 raft_consensus.cc:1275] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260807 19:50:11.843482 8740 leader_election.cc:400] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9.
I20260807 19:50:11.844120 12943 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 19:50:11.845332 13361 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:11.846367 8740 leader_election.cc:304] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:11.848763 13361 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:11.822064 13449 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:11.855355 13504 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.777354 13482 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:11.857034 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:11.866006 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:11.867071 13342 raft_consensus.cc:2749] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:11.869550 8778 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:11.871142 8740 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:11.873951 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 101998us
I20260807 19:50:11.871831 8790 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:11.883787 13342 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260807 19:50:11.888960 13471 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 1.043s user 0.008s sys 0.009s
I20260807 19:50:11.892005 13274 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:11.894807 8788 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51022 (request call id 1512) took 1387 ms (client timeout 1795 ms). Trace:
I20260807 19:50:11.894869 8783 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: []
W20260807 19:50:11.895416 8788 rpcz_store.cc:269] 0807 19:50:10.507451 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:10.523114 (+ 15663us) service_pool.cc:224] Handling call
0807 19:50:11.894530 (+1371416us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b540023e1d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:11.894711 (+ 181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2074710912}
I20260807 19:50:11.898088 8775 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 7. Ops: []
I20260807 19:50:11.873056 13482 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:11.902172 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 98472.7us
W20260807 19:50:11.903249 8913 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1704) took 2027 ms (client timeout 1960 ms). Trace:
W20260807 19:50:11.903719 8913 rpcz_store.cc:269] 0807 19:50:09.873649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:09.954428 (+ 80779us) service_pool.cc:224] Handling call
0807 19:50:11.900733 (+1946305us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54000a3f54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:11.900842 (+ 109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821588224}
I20260807 19:50:11.906790 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:11.907902 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:11.900246 13394 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:11.914489 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:11.919471 8972 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:11.953074 8790 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:11.956776 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.965252 8790 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:50:11.968130 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:11.972751 8790 raft_consensus.cc:3060] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 8
W20260807 19:50:11.990757 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:11.991525 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:12.064185 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:12.112273 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.149255 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.151238 8778 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 5.
W20260807 19:50:12.158561 8780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53512 (request call id 204) took 1654 ms (client timeout 2120 ms). Trace:
W20260807 19:50:12.149716 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 230327us
I20260807 19:50:11.914323 13237 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260807 19:50:12.159044 8780 rpcz_store.cc:269] 0807 19:50:10.504253 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:10.508508 (+ 4255us) service_pool.cc:224] Handling call
0807 19:50:12.158386 (+1649878us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b5400327b54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:12.158501 (+ 115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686060160}
I20260807 19:50:12.171403 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.915939 13466 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:12.178185 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:11.920785 12943 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:12.175143 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.185041 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.187675 8778 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:50:12.147131 13459 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:12.186962 8788 raft_consensus.cc:2376] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8. Re-sending same reply.
I20260807 19:50:12.189170 8972 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:12.163924 13202 raft_consensus.cc:2804] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:12.198815 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:12.170825 13237 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.228889 13183 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:12.184486 13466 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.149850 13394 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.236887 13183 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.266426 13183 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:12.266630 13280 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:12.267920 13280 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.268285 13280 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:12.281556 13482 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.197204 13519 consensus_queue.cc:1059] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.087s
I20260807 19:50:12.285346 13519 raft_consensus.cc:3055] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:12.285774 13519 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:12.286693 13519 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.289566 13394 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.241430 13466 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:12.320156 8788 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:12.320703 8788 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:12.321632 8788 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.322626 8788 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:12.324779 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.293308 8790 raft_consensus.cc:2468] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8.
I20260807 19:50:12.236685 13237 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:12.342432 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2093) took 1757 ms. Trace:
I20260807 19:50:12.344959 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.346134 8790 raft_consensus.cc:2393] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:12.343091 8789 rpcz_store.cc:276] 0807 19:50:10.585411 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.386241 (+800830us) service_pool.cc:224] Handling call
0807 19:50:11.575801 (+189560us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:11.582020 (+ 6219us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:12.342218 (+760198us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540040bad0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:12.342371 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435204864}
I20260807 19:50:12.350682 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.351480 8778 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:12.362748 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:12.363658 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.365622 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.371217 13183 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.230108 13459 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:12.381000 13183 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.382984 13459 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:12.383968 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.384491 8740 leader_election.cc:304] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.385339 13342 raft_consensus.cc:2749] T bf699a9379e64939a7048af91aef93ec P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:12.301301 12943 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260807 19:50:12.385035 13466 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.374666 13344 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.347739 13519 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:12.384511 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.388823 8915 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 19:50:12.389436 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.392704 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.393764 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.394174 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.394542 8778 raft_consensus.cc:2393] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 10: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:12.392925 13459 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:12.396862 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:12.397208 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.397045 8740 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.396279 13274 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:12.392570 13505 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.402374 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.404284 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.401782 13274 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.406749 8789 raft_consensus.cc:2393] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:12.394407 13342 raft_consensus.cc:2749] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:50:12.416064 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:12.418025 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 219260us
I20260807 19:50:12.419019 8972 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.419812 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2088) took 1879 ms. Trace:
I20260807 19:50:12.422046 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.422943 8790 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:12.420248 8783 rpcz_store.cc:276] 0807 19:50:10.540883 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.352847 (+811964us) service_pool.cc:224] Handling call
0807 19:50:11.894676 (+541829us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:11.895586 (+ 910us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:12.419583 (+523997us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b540023e1d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:12.419744 (+ 161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244068096}
I20260807 19:50:12.406042 13274 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:12.432811 8972 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.434974 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.438001 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.449790 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.451129 9018 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260807 19:50:12.451787 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.453506 9018 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260807 19:50:12.455017 8778 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:12.456645 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.458951 9023 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:12.402308 13505 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.459549 9017 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260807 19:50:12.460435 8906 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.020s sys 0.008s
W20260807 19:50:12.460662 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:12.462858 8906 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s user 0.020s sys 0.008s
W20260807 19:50:12.459919 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.456305 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:12.464432 8790 raft_consensus.cc:2368] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:12.465701 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.467490 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.475353 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.431190 13202 raft_consensus.cc:697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [term 10 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:12.425047 13466 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.407804 13459 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.455566 13183 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:12.474318 13342 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:12.494712 13455 log.cc:927] Time spent T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 1.947s user 0.009s sys 0.020s
W20260807 19:50:12.505246 8904 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54668 (request call id 1697) took 2688 ms (client timeout 2236 ms). Trace:
I20260807 19:50:12.477617 13393 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260807 19:50:12.515388 13537 log.cc:927] Time spent T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.147s user 0.000s sys 0.001s
W20260807 19:50:12.516866 13538 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.140s user 0.000s sys 0.001s
I20260807 19:50:12.456738 13501 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:12.532401 8788 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260807 19:50:12.533339 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 139945us
I20260807 19:50:12.534466 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.568118 8790 raft_consensus.cc:2393] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:12.569234 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.570436 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.570978 8790 raft_consensus.cc:2376] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8. Re-sending same reply.
W20260807 19:50:12.535737 8904 rpcz_store.cc:269] 0807 19:50:09.816790 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:09.824380 (+ 7590us) service_pool.cc:224] Handling call
0807 19:50:10.336013 (+511633us) spinlock_profiling.cc:243] Waited 1.46 ms on lock 0x7b58001e4b98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c689e 000000000038d209 00007f6880585940 00007f688064ded4 00007f688064df41 00007f68805fbeb0 00007f68805fb794 00007f6879b44834 00007f6879b447c2
0807 19:50:12.504961 (+2168948us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540020e754. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:12.505112 (+ 151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334708608}
I20260807 19:50:12.507085 8903 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:50:12.536545 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.538998 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.455065 13504 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
W20260807 19:50:12.542704 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 145760us
I20260807 19:50:12.524271 13493 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.575198 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.576696 8778 raft_consensus.cc:2393] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 10: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:12.577769 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:12.567384 8856 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:12.571817 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.582398 8790 raft_consensus.cc:2368] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 10. Current term is 11.
I20260807 19:50:12.566313 8788 pending_rounds.cc:85] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:50:12.584594 8788 pending_rounds.cc:107] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260807 19:50:12.585335 8788 raft_consensus.cc:2889] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:12.591615 8790 raft_consensus.cc:1275] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260807 19:50:12.596266 8788 raft_consensus.cc:3055] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:12.596807 8788 raft_consensus.cc:740] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:12.600378 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.603127 8789 raft_consensus.cc:1275] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260807 19:50:12.575397 13542 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:12.540107 13342 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.545848 13546 consensus_queue.cc:1048] T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260807 19:50:12.593494 13493 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:12.611276 9892 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260807 19:50:12.617159 9892 pending_rounds.cc:85] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Aborting all ops after (but not including) 7
I20260807 19:50:12.617460 9892 pending_rounds.cc:107] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260807 19:50:12.617727 9892 raft_consensus.cc:2889] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:12.618090 8788 consensus_queue.cc:260] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.613407 8778 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:12.624120 9020 raft_consensus.cc:1240] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:12.625178 13493 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:12.625994 13493 raft_consensus.cc:697] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:12.626540 13466 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.627240 13466 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.629220 8778 raft_consensus.cc:1275] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260807 19:50:12.629751 13466 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.627079 13493 consensus_queue.cc:237] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.624413 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:12.634261 9886 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Handling vote request from an unknown peer 671d8754170646eaac5748745eba58b4
I20260807 19:50:12.634981 9886 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 19:50:12.635818 13393 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.636509 13393 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.638993 13393 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.619939 13237 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.641064 9017 raft_consensus.cc:1240] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:12.643002 13237 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.643618 8788 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:12.644739 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.646067 13342 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.646533 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.607440 13183 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.647778 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.616873 13202 consensus_queue.cc:237] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.649056 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.649091 13125 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:12.615403 13542 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.656994 13542 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.657418 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2104) took 1390 ms. Trace:
I20260807 19:50:12.658236 13505 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:12.659729 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.605113 13280 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.663866 13280 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:12.639307 13274 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.669689 8787 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:12.671838 13274 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.676033 13459 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.681512 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.682202 9015 raft_consensus.cc:3055] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:12.682521 9015 raft_consensus.cc:740] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:12.683288 9015 consensus_queue.cc:260] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.684913 9027 raft_consensus.cc:2393] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:12.657845 8775 rpcz_store.cc:276] 0807 19:50:11.266789 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.819003 (+552214us) service_pool.cc:224] Handling call
0807 19:50:11.819307 (+ 304us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:11.898612 (+ 79305us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:12.657166 (+758554us) spinlock_profiling.cc:243] Waited 78.6 ms on lock 0x7b5400089dd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:12.657354 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180667264}
I20260807 19:50:12.685736 8790 raft_consensus.cc:1275] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:50:12.684226 9015 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:12.687444 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.692727 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:12.695634 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.698302 8788 raft_consensus.cc:1275] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260807 19:50:12.698765 8788 pending_rounds.cc:85] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 5
I20260807 19:50:12.705942 8788 pending_rounds.cc:107] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260807 19:50:12.706279 8788 raft_consensus.cc:2889] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:12.647701 13183 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.710371 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.715559 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:12.691120 13051 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.722640 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.698592 8849 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.690912 13505 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.699321 13459 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:12.726112 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:12.733886 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.736845 9017 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260807 19:50:12.742944 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.740700 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:12.705008 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 137712us
I20260807 19:50:12.739243 9008 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:12.745270 8856 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:12.746330 9023 raft_consensus.cc:2368] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
W20260807 19:50:12.750334 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51889.9us
I20260807 19:50:12.731307 13459 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.713433 13041 consensus_queue.cc:1059] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260807 19:50:12.754464 13560 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260807 19:50:12.756510 13567 raft_consensus.cc:2804] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:12.756964 13567 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:12.757306 13567 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:12.757524 13560 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:50:12.758255 13359 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.758975 13560 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:12.759378 13560 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:12.759694 13560 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:12.735450 13555 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.761204 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:12.778338 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 200627us
I20260807 19:50:12.780074 8740 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.764820 9015 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:12.781457 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.775012 8788 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
W20260807 19:50:12.796578 9838 leader_election.cc:336] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.221s (SENT)
I20260807 19:50:12.747481 12943 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.798374 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.799786 9838 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.801941 9838 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.738806 13505 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.749354 13504 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:12.761004 13459 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:12.805397 9838 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.732098 13051 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:12.783548 13065 consensus_queue.cc:1059] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260807 19:50:12.809633 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.803674 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.853991 9023 raft_consensus.cc:2368] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:50:12.805555 13359 raft_consensus.cc:2749] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260807 19:50:12.806032 13546 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:12.864136 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:12.862466 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:12.809019 13465 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260807 19:50:12.881098 13560 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.883644 13560 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:12.806085 13551 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:12.797703 13059 consensus_queue.cc:1048] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260807 19:50:12.852288 13495 raft_consensus.cc:3055] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:12.807938 13514 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:50:12.896171 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 86869.2us
W20260807 19:50:12.897940 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 115841us
I20260807 19:50:12.890911 13551 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:12.856310 13274 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.808508 13125 consensus_queue.cc:1048] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260807 19:50:12.806542 12943 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.901652 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:12.983518 8788 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
W20260807 19:50:12.949105 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46023.5us
I20260807 19:50:12.945886 8849 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:12.980513 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 235346us
I20260807 19:50:12.981308 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.985292 9023 raft_consensus.cc:1813] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Handling vote request from an unknown peer 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:12.985762 9023 raft_consensus.cc:2368] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 9.
I20260807 19:50:12.814210 13101 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260807 19:50:12.884832 13237 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.945256 13041 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
W20260807 19:50:12.988698 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.890929 13554 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.797784 13555 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:12.991158 8968 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:12.904618 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.813546 13504 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:12.938059 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.850167 13570 consensus_queue.cc:1048] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.133s
I20260807 19:50:12.866645 13342 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:13.017793 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.017997 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:12.903853 13546 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:12.947031 13565 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:12.866739 13493 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:12.895306 13495 raft_consensus.cc:740] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:13.012482 13519 consensus_meta.cc:287] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.620s user 0.010s sys 0.003s
I20260807 19:50:12.899933 13359 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:13.018368 13554 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:13.034449 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 280383us
I20260807 19:50:13.036023 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:13.040268 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 94455.9us
I20260807 19:50:13.041580 9023 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260807 19:50:13.042647 8849 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:12.896965 13560 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:12.990451 13065 consensus_queue.cc:1059] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260807 19:50:13.024855 13577 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:13.004168 13551 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:50:13.051942 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:12.991669 13274 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.044847 8849 leader_election.cc:304] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:13.011175 13237 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.026835 13546 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:13.063338 13466 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:13.068280 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.037614 13183 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:13.056632 13244 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:13.030237 13342 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:13.097453 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 52677.7us
I20260807 19:50:13.101837 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:13.064913 13577 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:13.107553 9023 raft_consensus.cc:2376] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10. Re-sending same reply.
I20260807 19:50:13.107751 8849 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.110426 13244 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.118458 13495 consensus_queue.cc:260] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.120052 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:13.121990 13567 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.124540 13577 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:13.125358 13555 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.126057 13555 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:13.137573 8849 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.189062 8788 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:13.192656 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.119024 13244 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:13.040429 13359 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.036579 13565 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.058928 13449 consensus_queue.cc:1048] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260807 19:50:13.102696 13546 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:13.119977 13495 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:13.062670 13560 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:13.051564 13493 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.239825 13568 raft_consensus.cc:2749] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:13.246548 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:13.343833 9023 raft_consensus.cc:3055] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:13.216305 13576 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.201s
I20260807 19:50:13.262883 13504 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.344249 9023 raft_consensus.cc:740] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:13.124226 13567 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.210208 13570 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260807 19:50:13.220502 13573 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
W20260807 19:50:13.267444 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:13.271800 13342 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.272503 13125 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:13.273036 13101 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:13.260223 13505 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:13.219660 13571 log.cc:927] Time spent T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.114s user 0.000s sys 0.000s
I20260807 19:50:13.366464 13125 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.273696 13274 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.274243 13051 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:13.274571 13587 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:13.368796 13125 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:13.366550 13101 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.369237 13587 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.369624 13582 consensus_queue.cc:1048] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260807 19:50:13.371026 13101 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:13.371587 13587 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.275605 13554 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.372018 13359 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.106851 13466 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.283393 13465 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:13.396242 13465 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:13.397713 13466 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:13.294776 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1717) took 2651 ms. Trace:
I20260807 19:50:13.300815 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:13.400111 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:13.400743 13505 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:13.401242 13505 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.401736 13465 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:13.294533 13593 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:13.320696 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 334059us
I20260807 19:50:13.406931 8856 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:13.322291 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 516961us
I20260807 19:50:13.323020 8788 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:13.344204 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:13.412657 8778 raft_consensus.cc:2376] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8. Re-sending same reply.
I20260807 19:50:13.413640 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.414592 13593 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:13.415482 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:13.415814 13593 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:13.416254 13593 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:13.416615 13593 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:13.417814 13568 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:50:13.419428 13568 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:13.420408 13551 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:13.420874 13449 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:13.422672 13551 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.414808 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.425207 8778 raft_consensus.cc:2410] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 19:50:13.426630 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:50:13.355340 8740 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.194:37659 timed out after 2.077s (SENT) [suppressed 58 similar messages]
W20260807 19:50:13.433656 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 78349.3us
I20260807 19:50:13.434620 13359 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:13.435230 13359 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:13.438854 13359 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:13.116156 13237 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.327792 13555 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:13.442526 8778 raft_consensus.cc:2376] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 8. Re-sending same reply.
I20260807 19:50:13.444024 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.390529 13493 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.393352 13560 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:13.394016 13504 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:13.450295 8778 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:13.301445 13183 raft_consensus.cc:993] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801: : Instructing follower 671d8754170646eaac5748745eba58b4 to start an election
I20260807 19:50:13.455056 13183 raft_consensus.cc:1081] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Signalling peer 671d8754170646eaac5748745eba58b4 to start an election
I20260807 19:50:13.456934 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.399125 8911 rpcz_store.cc:276] 0807 19:50:10.643412 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:10.643563 (+ 151us) service_pool.cc:224] Handling call
0807 19:50:12.582677 (+1939114us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b58001e4b98. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c689e 000000000038d209 00007f6880585940 00007f688064ded4 00007f688064df41 00007f688060fdd0 00007f68805fa739 00007f68805fa1e2 00007f6879b3fd34
0807 19:50:12.582879 (+ 202us) raft_consensus.cc:1471] Updating replica for 2 ops
0807 19:50:12.751572 (+168693us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:12.751609 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0807 19:50:12.804602 (+ 52993us) log.cc:844] Serialized 52 byte log entry
0807 19:50:13.021286 (+216684us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:13.210023 (+188737us) log.cc:844] Serialized 14 byte log entry
0807 19:50:13.213867 (+ 3844us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.213963 (+ 96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:13.294002 (+ 80039us) raft_consensus.cc:1685] finished
0807 19:50:13.294487 (+ 485us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:13.294658 (+ 171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16662,"prepare.queue_time_us":221222,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":315123,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":8429,"spinlock_wait_cycles":7489536,"thread_start_us":474115,"threads_started":3,"wal-append.queue_time_us":245451}
I20260807 19:50:13.404260 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:13.439029 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:13.347654 9023 consensus_queue.cc:260] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.446741 13237 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:13.378078 13565 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.403599 13505 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.459576 8788 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 10.
I20260807 19:50:13.308738 13459 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:13.461440 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:13.584271 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:13.585033 8778 raft_consensus.cc:2376] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6. Re-sending same reply.
W20260807 19:50:13.509505 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:13.585997 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:13.585928 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:13.595451 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 457949us
W20260807 19:50:13.526914 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:13.468979 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:13.598122 8783 raft_consensus.cc:2368] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:13.644022 13504 raft_consensus.cc:2749] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:13.477394 13577 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:13.703864 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 119487us
I20260807 19:50:13.704902 13459 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.705536 9023 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 7
I20260807 19:50:13.706818 13459 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:13.677873 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 267716us
I20260807 19:50:13.625838 13565 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:13.718916 13565 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:13.720019 13577 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:13.722314 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:13.723208 8911 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:13.724520 13237 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.726073 8778 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Handling vote request from an unknown peer 671d8754170646eaac5748745eba58b4
I20260807 19:50:13.726863 8778 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260807 19:50:13.709045 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 57678.4us
I20260807 19:50:13.447047 13546 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:13.729027 9892 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:13.709085 8788 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:13.731745 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2121) took 1341 ms. Trace:
I20260807 19:50:13.732100 8788 rpcz_store.cc:276] 0807 19:50:12.390625 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.708373 (+1317748us) service_pool.cc:224] Handling call
0807 19:50:13.708719 (+ 346us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.731417 (+ 22698us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.731675 (+ 258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.733786 8788 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:13.736380 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2123) took 1344 ms. Trace:
I20260807 19:50:13.734751 13546 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:13.747150 8849 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:13.782612 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 43969.7us
I20260807 19:50:13.785709 13459 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.788328 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:13.793351 8788 rpcz_store.cc:276] 0807 19:50:12.392117 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.733215 (+1341098us) service_pool.cc:224] Handling call
0807 19:50:13.733596 (+ 381us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.736057 (+ 2461us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.736289 (+ 232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.833467 8778 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:13.834483 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2122) took 1443 ms. Trace:
W20260807 19:50:13.835805 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 88779.5us
I20260807 19:50:13.836911 8849 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:13.868105 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 56652.3us
I20260807 19:50:13.834949 8778 rpcz_store.cc:276] 0807 19:50:12.390907 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.727724 (+1336817us) service_pool.cc:224] Handling call
0807 19:50:13.833044 (+105320us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.834144 (+ 1100us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.834416 (+ 272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.872243 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2128) took 1201 ms. Trace:
I20260807 19:50:13.872761 8778 rpcz_store.cc:276] 0807 19:50:12.670546 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.871352 (+1200806us) service_pool.cc:224] Handling call
0807 19:50:13.871712 (+ 360us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.871913 (+ 201us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:13.871938 (+ 25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:13.871984 (+ 46us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:50:13.872025 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.872076 (+ 51us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:13.872187 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.874707 8778 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260807 19:50:13.882505 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:13.883189 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 46380.2us
I20260807 19:50:13.884259 8849 leader_election.cc:400] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:13.884822 8849 leader_election.cc:403] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260807 19:50:13.880698 8778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2131) took 1209 ms. Trace:
I20260807 19:50:13.894903 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2120) took 1414 ms. Trace:
I20260807 19:50:13.895543 8783 rpcz_store.cc:276] 0807 19:50:12.390361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.643170 (+1252809us) service_pool.cc:224] Handling call
0807 19:50:13.744485 (+101315us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.744758 (+ 273us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:13.744785 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:13.744834 (+ 49us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:13.744903 (+ 69us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.744964 (+ 61us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:13.804828 (+ 59864us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.897243 8783 raft_consensus.cc:1275] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:50:13.897957 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2127) took 1227 ms. Trace:
I20260807 19:50:13.897826 8783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2133) took 1192 ms. Trace:
I20260807 19:50:13.899191 8783 rpcz_store.cc:276] 0807 19:50:12.705739 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.896576 (+1190837us) service_pool.cc:224] Handling call
0807 19:50:13.896903 (+ 327us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.897585 (+ 682us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.897768 (+ 183us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:50:13.898797 8740 leader_election.cc:336] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Timed out: RequestConsensusVote RPC to 127.6.42.195:35801 timed out after 1.665s (SENT)
I20260807 19:50:13.900404 8740 leader_election.cc:304] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.902089 8783 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 19:50:13.905776 8849 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.898419 8788 rpcz_store.cc:276] 0807 19:50:12.670251 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.897079 (+1226828us) service_pool.cc:224] Handling call
0807 19:50:13.897399 (+ 320us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.897603 (+ 204us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:13.897630 (+ 27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:13.897677 (+ 47us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:50:13.897718 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.897770 (+ 52us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:13.897894 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.907083 8778 rpcz_store.cc:276] 0807 19:50:12.671346 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.874028 (+1202682us) service_pool.cc:224] Handling call
0807 19:50:13.874343 (+ 315us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.880404 (+ 6061us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.880632 (+ 228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:13.940799 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:13.941488 8908 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:13.943349 8906 raft_consensus.cc:1275] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:50:13.944736 8849 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.945101 8902 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:13.945991 8849 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.947880 8901 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:13.950700 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.953253 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.951551 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.954627 8915 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 10: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:13.955298 8910 raft_consensus.cc:2410] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260807 19:50:13.956488 8906 raft_consensus.cc:2393] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
W20260807 19:50:13.958155 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 59403.4us
I20260807 19:50:13.960220 8902 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:13.870206 13546 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:13.962540 8849 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:13.938030 13429 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 7. Current term is 8.
I20260807 19:50:13.939778 13604 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:13.939216 13587 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:13.968005 9018 raft_consensus.cc:1240] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:13.969846 9027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57870 (request call id 1718) took 1535 ms. Trace:
I20260807 19:50:13.983413 13587 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:13.983850 9027 rpcz_store.cc:276] 0807 19:50:12.434569 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.434677 (+ 108us) service_pool.cc:224] Handling call
0807 19:50:13.967678 (+1533001us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400069fd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:13.969781 (+ 2103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522906496}
I20260807 19:50:13.985333 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:13.986629 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:13.987473 9027 raft_consensus.cc:2410] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260807 19:50:13.974974 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2097) took 3380 ms. Trace:
W20260807 19:50:13.977126 9008 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1263) took 2463 ms (client timeout 1726 ms). Trace:
I20260807 19:50:13.978499 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.989801 8906 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:13.989308 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.990701 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.991284 8906 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:13.950780 13555 raft_consensus.cc:2749] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:13.981408 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:13.992501 9008 rpcz_store.cc:269] 0807 19:50:11.506684 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.506793 (+ 109us) service_pool.cc:224] Handling call
0807 19:50:13.969262 (+2462469us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540034dfd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:13.969606 (+ 344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750822656}
I20260807 19:50:13.982715 8915 raft_consensus.cc:1275] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260807 19:50:13.973932 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:13.935168 13602 raft_consensus.cc:2749] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:13.966660 13560 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:14.004781 9023 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:13.953088 13495 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:13.999900 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:13.956461 13359 raft_consensus.cc:2749] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260807 19:50:13.988561 8776 rpcz_store.cc:276] 0807 19:50:10.594217 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.951012 (+3356795us) service_pool.cc:224] Handling call
0807 19:50:13.971760 (+ 20748us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.972125 (+ 365us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:13.972156 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:13.972206 (+ 50us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:50:13.972249 (+ 43us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:13.972332 (+ 83us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:13.974898 (+ 2566us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:50:14.012084 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.007486 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.009281 9008 raft_consensus.cc:2376] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8. Re-sending same reply.
W20260807 19:50:14.003199 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 175505us
I20260807 19:50:14.010434 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.011248 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.014272 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.023276 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:13.986797 13587 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.026336 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.030292 8902 raft_consensus.cc:2468] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:13.965284 13598 raft_consensus.cc:2749] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:14.034469 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.036624 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.040601 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.051925 13565 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:14.052114 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:13.974196 13546 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:14.021973 13603 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:14.056943 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.057682 9021 raft_consensus.cc:2368] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:50:14.064152 8901 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:14.067440 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.068063 8901 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:14.071734 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.073661 9008 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:50:14.078816 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 116317us
I20260807 19:50:14.060966 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.079846 8849 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.081314 8849 leader_election.cc:400] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 10. Current term is 11.
I20260807 19:50:14.081640 8849 leader_election.cc:403] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:50:14.081737 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.017210 13495 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.039714 13587 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:14.137406 8779 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:14.137751 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.122056 8910 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:14.211035 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2095) took 3619 ms. Trace:
I20260807 19:50:14.176587 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.213469 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.176927 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.193475 9008 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
W20260807 19:50:14.166167 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:14.225522 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.131414 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.213876 8779 rpcz_store.cc:276] 0807 19:50:10.591397 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:14.136750 (+3545353us) service_pool.cc:224] Handling call
0807 19:50:14.137201 (+ 451us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:14.210688 (+ 73487us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.210968 (+ 280us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:50:14.208139 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.064646 13565 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:14.094327 8849 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.228886 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:14.230897 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 215405us
I20260807 19:50:14.231388 9021 raft_consensus.cc:2410] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:50:14.233122 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:14.234664 8740 leader_election.cc:336] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Timed out: RequestConsensusVote RPC to 127.6.42.195:35801 timed out after 1.715s (SENT)
I20260807 19:50:14.236503 9008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f"
dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
mode: GRACEFUL
new_leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
from {username='slave'} at 127.0.0.1:55416
I20260807 19:50:14.235889 8740 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:14.236593 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 142334us
I20260807 19:50:14.237558 9008 raft_consensus.cc:606] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Received request to transfer leadership to TS 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:14.238389 9008 raft_consensus.cc:614] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260807 19:50:14.238920 8849 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.240209 8849 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.246623 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.092885 13504 raft_consensus.cc:2749] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 10. Current term is 11.
I20260807 19:50:14.137271 13560 consensus_queue.cc:1048] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260807 19:50:14.255646 9027 raft_consensus.cc:1240] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:14.257632 8908 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:14.093108 13598 raft_consensus.cc:2749] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:14.260756 8908 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260807 19:50:14.241724 13555 raft_consensus.cc:2804] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260807 19:50:14.276706 8911 consensus_meta.cc:287] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.553s user 0.008s sys 0.006s
I20260807 19:50:14.277197 8911 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:14.140869 13505 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.107s
I20260807 19:50:14.235001 13237 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.234804 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:14.259369 9027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 1294) took 1015 ms. Trace:
I20260807 19:50:14.268131 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.279611 9027 rpcz_store.cc:276] 0807 19:50:13.244237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:14.236726 (+992489us) service_pool.cc:224] Handling call
0807 19:50:14.255436 (+ 18710us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:14.259045 (+ 3609us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.259299 (+ 254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:14.280301 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1710) took 2923 ms. Trace:
I20260807 19:50:14.225338 13587 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.243669 13613 raft_consensus.cc:2749] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:14.280483 8910 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:50:14.241463 13565 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:14.085764 13603 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.287278 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.252812 13359 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:14.280865 9019 rpcz_store.cc:276] 0807 19:50:11.344107 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.344233 (+ 126us) service_pool.cc:224] Handling call
0807 19:50:11.344594 (+ 361us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:11.457350 (+112756us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:11.457401 (+ 51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:11.476725 (+ 19324us) log.cc:844] Serialized 26 byte log entry
0807 19:50:11.659479 (+182754us) raft_consensus.cc:1645] Marking committed up to 8
0807 19:50:11.682585 (+ 23106us) log.cc:844] Serialized 14 byte log entry
0807 19:50:11.682872 (+ 287us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.259043 (+2576171us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540002d354. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:14.259110 (+ 67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:14.259167 (+ 57us) raft_consensus.cc:1685] finished
0807 19:50:14.267174 (+ 8007us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:14.267343 (+ 169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45822,"prepare.queue_time_us":22895,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":4545,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":29761,"spinlock_wait_cycles":258401664,"thread_start_us":182337,"threads_started":1,"wal-append.queue_time_us":352746,"wal-append.run_cpu_time_us":2134,"wal-append.run_wall_time_us":1006996}
I20260807 19:50:14.250550 13459 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:14.305552 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 70925.8us
I20260807 19:50:14.269501 13555 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:14.279088 13429 raft_consensus.cc:2749] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:14.314291 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.317562 8911 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:14.321650 13587 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.322384 13587 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:50:14.332105 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.333137 9027 raft_consensus.cc:1240] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:50:14.305138 13504 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:14.333945 13504 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.345991 13565 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:14.361508 8910 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:14.367477 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.306505 13598 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:14.369248 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.370083 9019 raft_consensus.cc:2368] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:50:14.370307 9023 raft_consensus.cc:2393] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 6: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
W20260807 19:50:14.328111 13593 consensus_meta.cc:287] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.911s user 0.001s sys 0.007s
I20260807 19:50:14.371405 8740 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.372581 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.378940 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.379341 9027 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:14.373299 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.336232 13504 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.301610 9886 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:14.376704 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.404834 8911 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 11.
I20260807 19:50:14.377317 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.316807 13555 raft_consensus.cc:3060] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:14.382650 13598 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:14.382994 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.383317 13495 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:14.384318 13546 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.388129 13614 raft_consensus.cc:2749] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:14.388957 13603 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.389539 9890 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33434 (request call id 211) took 2614 ms. Trace:
I20260807 19:50:14.389955 8740 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.390452 13593 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:14.391052 13549 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260807 19:50:14.391968 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2096) took 3799 ms. Trace:
I20260807 19:50:14.420732 8787 rpcz_store.cc:276] 0807 19:50:10.592975 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.654042 (+2061067us) service_pool.cc:224] Handling call
0807 19:50:12.654492 (+ 450us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:12.670151 (+ 15659us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.391696 (+1721545us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b540008cfd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:14.391906 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34298240}
I20260807 19:50:14.397624 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1731) took 2635 ms. Trace:
I20260807 19:50:14.405977 8910 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:14.409435 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.409796 13550 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:14.412107 13604 raft_consensus.cc:2749] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:14.412786 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.420037 8737 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:14.426616 8911 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
W20260807 19:50:14.427157 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 195420us
I20260807 19:50:14.428491 8740 leader_election.cc:400] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8.
I20260807 19:50:14.429308 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:14.430152 8740 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.372159 13565 consensus_queue.cc:1059] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260807 19:50:14.421929 8903 rpcz_store.cc:276] 0807 19:50:11.762293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.866904 (+104611us) service_pool.cc:224] Handling call
0807 19:50:12.506672 (+639768us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:12.573216 (+ 66544us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.397376 (+1824160us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b540020e750. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:14.397550 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470640384}
I20260807 19:50:14.434775 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:14.435758 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.423184 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.436302 8913 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:14.438669 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.439344 9027 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 3.
I20260807 19:50:14.420733 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.440092 8904 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:50:14.441702 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.442330 9027 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:14.448374 8901 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:50:14.452919 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.422993 13614 raft_consensus.cc:2749] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:14.435372 8903 raft_consensus.cc:2376] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10. Re-sending same reply.
I20260807 19:50:14.465041 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.470486 8904 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
I20260807 19:50:14.420291 13560 raft_consensus.cc:493] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:14.470860 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.468600 9027 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:14.473598 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.470799 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.474298 9027 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 3.
I20260807 19:50:14.474570 8910 raft_consensus.cc:2393] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 10: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:14.432511 13603 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260807 19:50:14.464092 6315 auto_leader_rebalancer.cc:467] leader transfer for tablet d54b306954094de7a87c17dbbf6ef21f (from a38f4d6088cf4c8cb510200ca0555801 to 8e9ad2b5e2a94d4faff90af735c6d792) failed: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "not currently leader" }
I20260807 19:50:14.436265 13359 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader election won for term 10
W20260807 19:50:14.481228 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.485124 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.481397 6315 auto_leader_rebalancer.cc:474] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260807 19:50:14.484473 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.450564 13474 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:14.450600 13516 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:14.463922 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:50:14.493891 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 253754us
I20260807 19:50:14.495599 8849 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.501536 13404 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.501735 13598 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.493777 13474 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.481222 13603 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:14.476655 13241 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:14.501410 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 71316.5us
I20260807 19:50:14.505890 8740 leader_election.cc:400] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 5. Current term is 6.
I20260807 19:50:14.506655 8849 leader_election.cc:400] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:14.507133 8849 leader_election.cc:403] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:50:14.427614 13550 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:14.500850 13587 raft_consensus.cc:3055] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:14.508419 13241 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.418598 9890 rpcz_store.cc:276] 0807 19:50:11.775594 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:11.777987 (+ 2393us) service_pool.cc:224] Handling call
0807 19:50:11.778326 (+ 339us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:12.150246 (+371920us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:12.157356 (+ 7110us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:12.169930 (+ 12574us) raft_consensus.cc:1645] Marking committed up to 8
0807 19:50:12.174844 (+ 4914us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.367546 (+2192702us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54005affd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:14.372098 (+ 4552us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:14.389458 (+ 17360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314652032}
I20260807 19:50:14.518114 13474 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:14.503376 13565 raft_consensus.cc:2749] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:14.520599 8849 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:14.507261 13603 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:14.504688 13604 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:14.480490 13560 raft_consensus.cc:515] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.503927 13359 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:14.522272 8740 leader_election.cc:304] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.508085 13614 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:14.520599 13587 raft_consensus.cc:740] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:14.523924 13626 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185) with higher term. Message: Invalid argument: T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9.
I20260807 19:50:14.592111 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.534919 13614 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:14.595319 13614 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:14.536561 13237 raft_consensus.cc:2749] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:50:14.537340 8740 leader_election.cc:400] T 300f918f185249e6895ab57edd060d31 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 9.
I20260807 19:50:14.597416 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.598845 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:14.597316 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 60122us
I20260807 19:50:14.599444 8902 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:14.540980 8904 raft_consensus.cc:1240] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:14.542092 8910 raft_consensus.cc:1240] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:14.544564 8903 raft_consensus.cc:1275] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:14.602341 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.602985 8903 raft_consensus.cc:2376] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10. Re-sending same reply.
I20260807 19:50:14.603892 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.545171 13549 consensus_queue.cc:1048] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:14.545032 13359 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.551707 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.553213 13593 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.554343 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.530485 13613 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:14.558034 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.614602 8913 raft_consensus.cc:2393] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 6: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:14.558349 13560 leader_election.cc:290] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.559367 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:14.560333 9020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1717) took 2126 ms. Trace:
I20260807 19:50:14.619169 9023 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:14.536468 9885 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33434 (request call id 214) took 1759 ms. Trace:
I20260807 19:50:14.577122 8856 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.577946 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:14.579236 13323 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260807 19:50:14.581979 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53512 (request call id 211) took 2630 ms. Trace:
I20260807 19:50:14.627404 8780 rpcz_store.cc:276] 0807 19:50:11.951222 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.179915 (+228693us) service_pool.cc:224] Handling call
0807 19:50:12.205069 (+ 25154us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:12.231043 (+ 25974us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:12.231077 (+ 34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:12.234984 (+ 3907us) log.cc:844] Serialized 26 byte log entry
0807 19:50:12.430533 (+195549us) raft_consensus.cc:1645] Marking committed up to 9
0807 19:50:12.689843 (+259310us) log.cc:844] Serialized 14 byte log entry
0807 19:50:14.581408 (+1891565us) spinlock_profiling.cc:243] Waited 59.5 ms on lock 0x7b5400096120. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b57fb13 00007f687b60519f 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c
0807 19:50:14.581470 (+ 62us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.581577 (+ 107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:14.581626 (+ 49us) raft_consensus.cc:1685] finished
0807 19:50:14.581725 (+ 99us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:14.581871 (+ 146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":324094,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":6296,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":9237,"spinlock_wait_cycles":136941440,"thread_start_us":420386,"threads_started":2,"wal-append.queue_time_us":351802,"wal-append.run_cpu_time_us":4388,"wal-append.run_wall_time_us":1297693}
I20260807 19:50:14.629966 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.630700 8780 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:14.582428 13344 raft_consensus.cc:2749] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:14.562161 13587 consensus_queue.cc:260] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.583845 8968 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:14.634501 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 50778.5us
I20260807 19:50:14.635914 8968 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:14.586990 9017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 1270) took 1944 ms. Trace:
I20260807 19:50:14.588099 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.588939 13241 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:14.589408 9018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 1272) took 1944 ms. Trace:
I20260807 19:50:14.540205 13555 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:14.642050 9018 rpcz_store.cc:276] 0807 19:50:12.645250 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.645407 (+ 157us) service_pool.cc:224] Handling call
0807 19:50:13.967809 (+1322402us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.983435 (+ 15626us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.589215 (+605780us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400069fd0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:14.589347 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3040584960}
I20260807 19:50:14.573382 13603 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:14.605093 8903 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:14.606911 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.608078 8972 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:14.609953 8911 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:14.610080 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.612712 8740 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:14.610790 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:14.621470 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 43589.3us
I20260807 19:50:14.619171 9020 rpcz_store.cc:276] 0807 19:50:12.434213 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.434385 (+ 172us) service_pool.cc:224] Handling call
0807 19:50:12.453993 (+ 19608us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:12.624631 (+170638us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.560079 (+1935448us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540043c1d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:14.560265 (+ 186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390804224}
I20260807 19:50:14.586520 13404 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:14.636693 13394 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260807 19:50:14.639806 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 62799.8us
I20260807 19:50:14.670917 8856 leader_election.cc:400] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:50:14.671360 8856 leader_election.cc:403] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260807 19:50:14.641543 13480 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:50:14.646260 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:14.591514 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.645571 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.641491 9017 rpcz_store.cc:276] 0807 19:50:12.642720 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.642847 (+ 127us) service_pool.cc:224] Handling call
0807 19:50:12.643506 (+ 659us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:12.741441 (+ 97935us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.586695 (+1845254us) spinlock_profiling.cc:243] Waited 93 ms on lock 0x7b5400380754. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:14.586906 (+ 211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213859200}
I20260807 19:50:14.632007 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.650281 8788 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:14.715655 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:14.716403 8788 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:14.714449 8780 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:14.690258 8856 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:14.506064 13516 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:14.723598 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 115630us
W20260807 19:50:14.723634 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 53208.7us
W20260807 19:50:14.724423 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 52127.1us
I20260807 19:50:14.714002 8902 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:14.731591 8911 raft_consensus.cc:2468] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:14.739835 9023 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:14.741462 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.742281 8901 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:14.743777 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.749537 9892 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33452 (request call id 338) took 1710 ms. Trace:
I20260807 19:50:14.749867 8856 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:14.638782 13202 raft_consensus.cc:3055] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:14.752815 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:14.748386 8789 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:14.767529 8856 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:14.873764 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 106313us
I20260807 19:50:14.751610 9892 rpcz_store.cc:276] 0807 19:50:13.039021 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:13.039170 (+ 149us) service_pool.cc:224] Handling call
0807 19:50:13.039537 (+ 367us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:13.729447 (+689910us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.749276 (+1019829us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b5400565ed4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:14.749472 (+ 196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1585216384}
I20260807 19:50:14.669302 13519 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:14.706827 13394 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:14.890473 8788 raft_consensus.cc:2468] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:14.898660 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.899713 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.896123 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.905614 8904 raft_consensus.cc:2393] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 8: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:14.707767 13480 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:14.746702 13593 raft_consensus.cc:2749] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:14.747198 13560 raft_consensus.cc:3060] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:14.907097 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:14.923606 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.881947 13598 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:50:14.758275 13202 raft_consensus.cc:740] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:14.756096 13504 raft_consensus.cc:2804] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:14.754573 13577 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:50:14.917466 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:14.928134 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 315491us
I20260807 19:50:14.903132 13613 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:14.753842 13059 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.167s
I20260807 19:50:14.896025 13519 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:14.707484 13404 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:14.906674 13394 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:14.744495 13241 consensus_queue.cc:1059] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.175s
I20260807 19:50:14.933705 8740 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:14.900892 13608 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:14.932461 8789 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:14.723222 13626 consensus_queue.cc:1059] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.198s
W20260807 19:50:14.929384 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 204312us
I20260807 19:50:14.918329 13480 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
W20260807 19:50:14.938730 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:14.871732 13618 consensus_queue.cc:1059] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.206s
I20260807 19:50:14.914031 13237 raft_consensus.cc:2804] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:14.945329 12927 consensus_queue.cc:1059] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260807 19:50:14.974126 8740 leader_election.cc:400] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:50:14.973954 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:14.936616 13577 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260807 19:50:14.990361 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 115383us
I20260807 19:50:14.921998 13549 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260807 19:50:14.936199 13504 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:14.930234 13598 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:50:15.028693 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 304082us
I20260807 19:50:14.999665 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:15.000643 8778 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:14.968879 13459 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:14.985819 8789 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260807 19:50:15.085673 8789 pending_rounds.cc:85] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:50:15.085959 8789 pending_rounds.cc:107] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260807 19:50:14.996446 13608 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:15.032882 8901 raft_consensus.cc:2376] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7. Re-sending same reply.
I20260807 19:50:15.033550 13618 consensus_queue.cc:1048] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260807 19:50:15.037137 13631 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
W20260807 19:50:15.037992 13630 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.331s user 0.001s sys 0.000s
I20260807 19:50:15.039743 13555 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:14.967659 13613 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:15.107335 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:14.624457 9885 rpcz_store.cc:276] 0807 19:50:12.777051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.780928 (+ 3877us) service_pool.cc:224] Handling call
0807 19:50:12.855283 (+ 74355us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:14.051852 (+1196569us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b18001023a0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f687b1ab6ac 00007f68784e0e3b 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fae05 00007f687b5fc440 00007f687b5fcfdb 00007f687b607a2b
0807 19:50:14.052052 (+ 200us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:14.052085 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:14.052769 (+ 684us) log.cc:844] Serialized 26 byte log entry
0807 19:50:14.409624 (+356855us) raft_consensus.cc:1645] Marking committed up to 7
0807 19:50:14.409675 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:14.410218 (+ 543us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:14.536105 (+125887us) raft_consensus.cc:1685] finished
0807 19:50:14.536130 (+ 25us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:14.536380 (+ 250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121375,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2835,"spinlock_wait_cycles":12649856,"thread_start_us":470946,"threads_started":2,"wal-append.queue_time_us":430500}
I20260807 19:50:15.086808 13237 raft_consensus.cc:697] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
W20260807 19:50:15.084470 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 110437us
I20260807 19:50:15.122255 8740 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:15.086200 8789 raft_consensus.cc:2889] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:15.040055 13241 raft_consensus.cc:3055] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:14.982441 13629 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:15.130198 13629 consensus_queue.cc:1059] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260807 19:50:15.130414 13613 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.132306 13647 raft_consensus.cc:3055] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:15.132745 13647 raft_consensus.cc:740] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:15.133734 13647 consensus_queue.cc:260] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.135063 13647 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:15.135624 13608 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:15.040947 13538 log.cc:927] Time spent T 7e8d547d81964989bb6cd1b677c971c4 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20260807 19:50:15.136972 13645 raft_consensus.cc:2749] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:50:15.083537 13504 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.125409 13241 raft_consensus.cc:740] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:15.138681 13241 consensus_queue.cc:260] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.139899 13241 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:15.086390 13598 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:15.086249 13614 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:14.948844 13519 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:15.146226 13504 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.147917 13404 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:15.148574 13404 raft_consensus.cc:3055] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:15.148969 13404 raft_consensus.cc:740] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:15.149859 13404 consensus_queue.cc:260] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.089143 13577 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:15.148289 8906 raft_consensus.cc:2393] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 9: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:15.151559 13516 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:15.118091 8911 raft_consensus.cc:1275] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260807 19:50:15.144616 12943 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:15.129261 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 129667us
I20260807 19:50:15.154749 8972 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:15.156701 12943 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:15.157092 8902 raft_consensus.cc:1275] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:50:15.157233 12943 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:15.157737 12943 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.144670 13614 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.041502 8788 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:15.160845 6315 auto_leader_rebalancer.cc:844] All tables' leader rebalancing finished this round
W20260807 19:50:15.162410 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:15.171700 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.172645 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:15.185091 13637 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
I20260807 19:50:15.152182 13644 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader election won for term 4
W20260807 19:50:15.192723 8903 consensus_meta.cc:287] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.543s user 0.010s sys 0.000s
I20260807 19:50:15.196192 8903 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:15.202201 8903 raft_consensus.cc:1275] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:15.092411 13459 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:15.213575 8740 connection.cc:256] RPC call timeout handler was delayed by -1.2914945699740201s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
W20260807 19:50:15.186126 13631 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.071s user 0.001s sys 0.000s
I20260807 19:50:15.086215 13202 consensus_queue.cc:260] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:15.164829 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 80449.8us
W20260807 19:50:15.218930 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.217321 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.217031 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.226052 9018 raft_consensus.cc:2410] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 19:50:15.174221 13603 leader_election.cc:290] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:15.220378 8910 raft_consensus.cc:1240] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:15.176558 13608 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260807 19:50:15.274573 8910 raft_consensus.cc:1275] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260807 19:50:15.271567 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:15.278658 8968 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:15.174961 13565 consensus_queue.cc:1059] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260807 19:50:15.263454 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.248651 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.286772 9017 raft_consensus.cc:2376] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10. Re-sending same reply.
I20260807 19:50:15.160739 13394 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.207648 13642 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260807 19:50:15.276729 13202 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.292668 8968 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:15.250521 8911 raft_consensus.cc:1240] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:15.309804 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.281925 13549 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:15.300206 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
W20260807 19:50:15.312964 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 90867.7us
I20260807 19:50:15.269984 13543 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260807 19:50:15.305114 8968 leader_election.cc:304] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:15.317806 8968 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:15.294245 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 206472us
I20260807 19:50:15.313656 13059 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:15.298929 12927 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:15.282091 13086 consensus_queue.cc:1059] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
W20260807 19:50:15.317970 8740 connection.cc:256] RPC call timeout handler was delayed by -1.1744945698883384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
I20260807 19:50:15.313546 13549 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:15.312585 13565 raft_consensus.cc:3055] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
W20260807 19:50:15.321772 13563 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 2.474s user 0.015s sys 0.020s
W20260807 19:50:15.354689 13651 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.074s user 0.000s sys 0.000s
I20260807 19:50:15.355257 13603 raft_consensus.cc:493] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:15.339578 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:15.340292 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.286202 13237 consensus_queue.cc:237] T b8f9a221ee034d3b86101183048438ff P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.349961 13645 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260807 19:50:15.338562 13059 raft_consensus.cc:493] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:15.343580 13448 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:15.321251 13644 raft_consensus.cc:697] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:15.354087 9023 raft_consensus.cc:1275] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260807 19:50:15.351411 8740 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.373164 9018 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:15.350124 13549 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:15.373095 13059 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.334910 13504 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.342008 12927 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:15.321960 13660 raft_consensus.cc:2804] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:15.354111 13394 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:15.350538 13565 raft_consensus.cc:740] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:15.355832 13603 raft_consensus.cc:515] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.373314 13448 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
W20260807 19:50:15.427285 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 86283.7us
I20260807 19:50:15.373313 13459 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.436239 8902 raft_consensus.cc:1240] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:15.398412 12927 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:50:15.447531 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 68043.5us
W20260807 19:50:15.455458 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.456313 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.457620 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:15.458184 8783 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.461228 8788 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:15.463667 9015 raft_consensus.cc:2368] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 LEADER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 9. Current term is 10.
W20260807 19:50:15.464208 9015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57884 (request call id 1284) took 2702 ms (client timeout 2217 ms). Trace:
W20260807 19:50:15.464603 9015 rpcz_store.cc:269] 0807 19:50:12.761887 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:12.803405 (+ 41518us) service_pool.cc:224] Handling call
0807 19:50:15.463399 (+2659994us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400260320. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b57c1a0 00007f687b6081f8 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6
0807 19:50:15.464132 (+ 733us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401254784}
I20260807 19:50:15.178211 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:50:15.478683 8906 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703"
dest_uuid: "671d8754170646eaac5748745eba58b4"
from {username='slave'} at 127.0.0.1:54668
I20260807 19:50:15.479363 8906 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:50:15.479732 8906 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:15.496645 13344 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:15.497289 13344 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:15.499197 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:15.499805 8740 connection.cc:256] RPC call timeout handler was delayed by -1.0232945699244738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
I20260807 19:50:15.500574 8629 catalog_manager.cc:5697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:50:15.502151 8629 catalog_manager.cc:5697] T 138f0e8a31994c549dd743c9ba49d6ee P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 8 to 9, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196). New cstate: current_term: 9 leader_uuid: "39975243456d4452b4a4870ac7ebafc5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260807 19:50:15.503739 8629 catalog_manager.cc:5697] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: term changed from 7 to 9, leader changed from 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196) to 671d8754170646eaac5748745eba58b4 (127.6.42.194). New cstate: current_term: 9 leader_uuid: "671d8754170646eaac5748745eba58b4" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:50:15.505113 8629 catalog_manager.cc:5697] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 reported cstate change: config changed from index 4 to 6, VOTER 671d8754170646eaac5748745eba58b4 (127.6.42.194) evicted. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } } }
I20260807 19:50:15.508744 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.510077 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.511139 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.514887 13550 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.517877 13642 raft_consensus.cc:493] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:15.518450 13642 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.519949 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:15.522137 13404 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:15.525314 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.525360 13587 raft_consensus.cc:3060] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.528582 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.529867 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:15.530953 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:15.531441 8787 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:15.532809 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:15.534091 13642 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.535173 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.535851 8788 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 6.
I20260807 19:50:15.541095 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.541852 8788 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 10.
I20260807 19:50:15.499418 13344 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.537698 9020 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
I20260807 19:50:15.512217 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.549167 8779 raft_consensus.cc:2393] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:15.553972 8630 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38434 (request call id 182) took 6916 ms. Trace:
I20260807 19:50:15.561751 13344 consensus_queue.cc:1048] T b89518940e0d4f7c848bc64e87de7e25 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260807 19:50:15.572477 8737 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.574388 8740 leader_election.cc:304] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:15.575558 13344 raft_consensus.cc:2749] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:15.457710 8856 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:15.577075 8902 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:15.579747 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1767) took 1029 ms. Trace:
I20260807 19:50:15.580331 8902 rpcz_store.cc:276] 0807 19:50:14.550450 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.437286 (+886836us) service_pool.cc:224] Handling call
0807 19:50:15.576873 (+139587us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.579465 (+ 2592us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.579684 (+ 219us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260807 19:50:15.585481 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 104810us
I20260807 19:50:15.425585 13660 raft_consensus.cc:697] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:15.394024 13577 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.593322 8972 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:50:15.609594 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.610999 8972 leader_election.cc:304] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:50:15.587142 13678 log.cc:927] Time spent T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.151s user 0.000s sys 0.000s
I20260807 19:50:15.438172 13448 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:15.601366 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:15.577194 13671 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.125s user 0.000s sys 0.000s
W20260807 19:50:15.597676 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 140055us
I20260807 19:50:15.612946 8972 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:15.631073 8972 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:15.618150 13086 raft_consensus.cc:2804] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:15.618651 13645 raft_consensus.cc:2804] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election won for term 8
W20260807 19:50:15.618867 13655 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.261s user 0.000s sys 0.000s
I20260807 19:50:15.621622 8902 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:15.436290 13519 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.374158 13644 consensus_queue.cc:237] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.637869 9020 raft_consensus.cc:1240] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260807 19:50:15.645156 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:15.646150 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1777) took 1092 ms. Trace:
I20260807 19:50:15.646278 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.646607 8902 rpcz_store.cc:276] 0807 19:50:14.553309 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.591890 (+1038581us) service_pool.cc:224] Handling call
0807 19:50:15.621424 (+ 29534us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.645891 (+ 24467us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.646090 (+ 199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:15.598763 13614 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.650251 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:15.654234 9892 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:15.655839 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:15.656672 9020 raft_consensus.cc:2393] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:15.670159 9021 raft_consensus.cc:1240] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:15.672802 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:15.618193 13592 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:15.674324 8902 raft_consensus.cc:1275] T 537cbacddca2410db509aff60875a134 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260807 19:50:15.675036 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1779) took 1121 ms. Trace:
I20260807 19:50:15.675437 8902 rpcz_store.cc:276] 0807 19:50:14.553846 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.673654 (+1119808us) service_pool.cc:224] Handling call
0807 19:50:15.674010 (+ 356us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.674765 (+ 755us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.674980 (+ 215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260807 19:50:15.634384 13516 raft_consensus.cc:2749] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:15.676589 13394 consensus_queue.cc:1048] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:15.678444 13645 raft_consensus.cc:697] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:15.679437 13645 consensus_queue.cc:237] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.680848 13603 leader_election.cc:290] T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.681566 13474 raft_consensus.cc:3055] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:15.682054 13474 raft_consensus.cc:740] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:15.682946 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:15.674724 13592 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:15.634690 13086 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:15.628655 13593 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:15.704447 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:15.714812 9892 raft_consensus.cc:2468] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:15.739691 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:15.743867 8903 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1760) took 1196 ms. Trace:
I20260807 19:50:15.745224 8903 rpcz_store.cc:276] 0807 19:50:14.547422 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.252062 (+704640us) service_pool.cc:224] Handling call
0807 19:50:15.252489 (+ 427us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.252740 (+ 251us) raft_consensus.cc:1534] Early marking committed up to index 6
0807 19:50:15.289882 (+ 37142us) log.cc:844] Serialized 14 byte log entry
0807 19:50:15.517330 (+227448us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800470a40. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f687b1ab6ac 00007f68784e0e3b 00007f6881388fc9 00007f68784fcfdb 00007f68784fb2db 00007f68784fd542 00007f687b19d3fb 00007f687b1a2f13 00007f687b1a342a
0807 19:50:15.743506 (+226176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:15.743562 (+ 56us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:50:15.743610 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.743674 (+ 64us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:15.743804 (+ 130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92968,"prepare.queue_time_us":38086,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":2547,"raft.queue_time_us":98945,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":14514,"spinlock_wait_cycles":6004992,"thread_start_us":177508,"threads_started":2,"wal-append.queue_time_us":130225}
I20260807 19:50:15.674363 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:15.780280 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.703507 13086 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:15.684284 13682 raft_consensus.cc:2749] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:15.766928 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:15.800562 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 228285us
I20260807 19:50:15.797999 8783 raft_consensus.cc:2468] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:15.814131 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:15.703889 13652 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:15.705863 13102 consensus_queue.cc:1048] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260807 19:50:15.610301 13565 consensus_queue.cc:260] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:15.855773 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 219915us
I20260807 19:50:15.856814 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2143) took 1245 ms. Trace:
I20260807 19:50:15.570520 8906 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.857787 8972 leader_election.cc:304] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:15.760596 13516 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:15.802662 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:15.862910 8783 raft_consensus.cc:1240] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260807 19:50:15.845193 8776 raft_consensus.cc:2468] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:15.799387 8903 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 8.
W20260807 19:50:15.798628 13671 log.cc:927] Time spent T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.073s user 0.000s sys 0.000s
I20260807 19:50:15.848927 8778 raft_consensus.cc:2410] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:50:15.846946 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.731174 13394 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:15.554579 8630 rpcz_store.cc:276] 0807 19:50:08.637749 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:08.638014 (+ 265us) service_pool.cc:224] Handling call
0807 19:50:08.738444 (+100430us) spinlock_profiling.cc:243] Waited 92.4 ms on lock 0x7b5800001420. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6878386981 00007f68784c6521 00007f68784c6a62 00007f6881492df9 00007f688142bd56 00007f68813fd248 00007f68814f9bfe 00007f687f9b34a4 00007f687f9b3432 00007f687f9b3361 00007f687f9b32dc
0807 19:50:15.553909 (+6815465us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:50:11.914545 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:50:12.276625 (+362080us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:50:12.276668 (+ 43us) write_op.cc:435] Acquired schema lock
0807 19:50:12.276686 (+ 18us) tablet.cc:662] Decoding operations
0807 19:50:12.277984 (+ 1298us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:50:12.278076 (+ 92us) write_op.cc:641] Partition lock acquired for write op
0807 19:50:12.278103 (+ 27us) tablet.cc:685] Acquiring locks for 14 operations
0807 19:50:12.279175 (+ 1072us) tablet.cc:701] Row locks acquired
0807 19:50:12.279197 (+ 22us) write_op.cc:260] PREPARE: finished
0807 19:50:12.279383 (+ 186us) write_op.cc:270] Start()
0807 19:50:12.279513 (+ 130us) write_op.cc:276] Timestamp: P: 1786132212279329 usec, L: 0
0807 19:50:12.279536 (+ 23us) op_driver.cc:348] REPLICATION: starting
0807 19:50:12.280322 (+ 786us) log.cc:844] Serialized 5540 byte log entry
0807 19:50:13.034468 (+754146us) op_driver.cc:464] REPLICATION: finished
0807 19:50:13.464377 (+429909us) write_op.cc:301] APPLY: starting
0807 19:50:13.464483 (+ 106us) tablet.cc:1367] starting BulkCheckPresence
0807 19:50:13.464799 (+ 316us) tablet.cc:1370] finished BulkCheckPresence
0807 19:50:13.464933 (+ 134us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:50:13.466667 (+ 1734us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:50:13.466961 (+ 294us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0807 19:50:13.467009 (+ 48us) write_op.cc:312] APPLY: finished
0807 19:50:13.467401 (+ 392us) log.cc:844] Serialized 101 byte log entry
0807 19:50:13.654623 (+187222us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:13.746606 (+ 91983us) write_op.cc:454] Released schema lock
0807 19:50:13.746661 (+ 55us) write_op.cc:341] FINISH: Updating metrics
0807 19:50:13.949417 (+202756us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:13.953838 (+ 4421us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17167,"reported_tablets":41,"spinlock_wait_cycles":212587392,"child_traces":[["op",{"apply.queue_time_us":421507,"apply.run_cpu_time_us":8604,"apply.run_wall_time_us":432739,"num_ops":14,"prepare.queue_time_us":149491,"prepare.run_cpu_time_us":51998,"prepare.run_wall_time_us":643422,"raft.queue_time_us":263254,"raft.run_cpu_time_us":14111,"raft.run_wall_time_us":355676,"replication_time_us":736492,"thread_start_us":676246,"threads_started":4,"wal-append.queue_time_us":296256,"wal-append.run_cpu_time_us":11148,"wal-append.run_wall_time_us":1834462}]]}
I20260807 19:50:15.834656 8737 leader_election.cc:304] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:15.879742 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.860069 8904 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:15.862460 8740 leader_election.cc:304] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:15.865154 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:15.857239 8780 rpcz_store.cc:276] 0807 19:50:14.610961 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:14.922212 (+311251us) service_pool.cc:224] Handling call
0807 19:50:14.936990 (+ 14778us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:14.948564 (+ 11574us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:14.948601 (+ 37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:14.949176 (+ 575us) log.cc:844] Serialized 26 byte log entry
0807 19:50:15.186496 (+237320us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:15.186541 (+ 45us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.186616 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:15.856549 (+669933us) raft_consensus.cc:1685] finished
0807 19:50:15.856573 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:15.856720 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11082,"raft.queue_time_us":11551,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":19415,"thread_start_us":216542,"threads_started":1,"wal-append.queue_time_us":315530}
I20260807 19:50:15.880488 8908 raft_consensus.cc:2376] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10. Re-sending same reply.
I20260807 19:50:15.806774 8787 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:15.881958 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.882885 8908 raft_consensus.cc:2410] T 7e8d547d81964989bb6cd1b677c971c4 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260807 19:50:15.883714 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea5ba8c8385a404799a0be85834f98f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:15.884927 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 65047.3us
I20260807 19:50:15.888609 8740 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:15.897413 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1781) took 1335 ms. Trace:
I20260807 19:50:15.894405 8740 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:15.898124 8913 rpcz_store.cc:276] 0807 19:50:14.562076 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.678521 (+1116445us) service_pool.cc:224] Handling call
0807 19:50:15.678849 (+ 328us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.679063 (+ 214us) raft_consensus.cc:1534] Early marking committed up to index 5
0807 19:50:15.679345 (+ 282us) log.cc:844] Serialized 14 byte log entry
0807 19:50:15.897045 (+217700us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:15.897110 (+ 65us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:15.897157 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.897223 (+ 66us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:15.897352 (+ 129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134487,"prepare.queue_time_us":153,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"spinlock_wait_cycles":1920,"thread_start_us":82791,"threads_started":1,"wal-append.queue_time_us":183155}
I20260807 19:50:15.899132 8740 leader_election.cc:304] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:15.899974 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.900964 8913 raft_consensus.cc:2468] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:50:15.894794 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.729943 13474 consensus_queue.cc:260] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.902477 8740 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:15.889045 9885 raft_consensus.cc:1240] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:15.912405 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.896895 13404 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:50:15.973104 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 106637us
W20260807 19:50:15.973258 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 195938us
I20260807 19:50:15.974139 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1765) took 1424 ms. Trace:
I20260807 19:50:15.978097 8910 rpcz_store.cc:276] 0807 19:50:14.549865 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.275970 (+726105us) service_pool.cc:224] Handling call
0807 19:50:15.276412 (+ 442us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.280598 (+ 4186us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 19:50:15.395842 (+115244us) log.cc:844] Serialized 14 byte log entry
0807 19:50:15.973729 (+577887us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:15.973797 (+ 68us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:15.973852 (+ 55us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.973944 (+ 92us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:15.974077 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121356,"prepare.queue_time_us":44439,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":3287,"thread_start_us":564588,"threads_started":2}
I20260807 19:50:15.893079 13614 raft_consensus.cc:2804] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election won for term 7
W20260807 19:50:15.979699 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 87019.9us
I20260807 19:50:15.980188 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:15.980607 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:15.981247 9022 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 9.
I20260807 19:50:15.853974 13652 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.903923 13613 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:15.905965 9890 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:15.903373 13629 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader election won for term 8
I20260807 19:50:15.906751 13642 raft_consensus.cc:2804] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:15.989127 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:15.887497 13394 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:15.896883 13344 consensus_queue.cc:1048] T d54b306954094de7a87c17dbbf6ef21f P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.093s
I20260807 19:50:15.880468 13516 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.833642 13660 consensus_queue.cc:237] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:15.903072 13593 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:16.003293 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:16.004083 13504 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:16.006621 13698 log.cc:927] Time spent T ef1e484df4724d9896a8f46b89923261 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.096s user 0.001s sys 0.000s
I20260807 19:50:15.914547 13674 raft_consensus.cc:2749] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260807 19:50:16.005700 13480 consensus_meta.cc:287] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.978s user 0.007s sys 0.005s
I20260807 19:50:16.049288 8910 raft_consensus.cc:2468] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 5.
I20260807 19:50:15.994980 13614 raft_consensus.cc:697] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:16.035868 8915 raft_consensus.cc:2468] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:50:15.988248 13359 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260807 19:50:16.052860 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 195162us
W20260807 19:50:16.064708 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 81234.3us
I20260807 19:50:16.007800 13592 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:15.998416 13577 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:16.063844 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:16.076987 8910 raft_consensus.cc:2468] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:16.071591 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:16.074241 13614 consensus_queue.cc:237] T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:16.031018 13519 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:16.016506 13642 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:16.069188 13598 consensus_meta.cc:287] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.928s user 0.007s sys 0.004s
I20260807 19:50:16.059136 13604 raft_consensus.cc:493] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:15.997354 13693 consensus_queue.cc:1059] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260807 19:50:16.067162 13216 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.021780 13613 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:15.999051 13697 consensus_queue.cc:1059] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
I20260807 19:50:16.067261 13480 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:16.067623 13703 raft_consensus.cc:2804] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:16.066009 13504 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.098162 13661 raft_consensus.cc:493] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:16.098708 13059 consensus_meta.cc:287] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.708s user 0.007s sys 0.004s
I20260807 19:50:16.150283 8911 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1750) took 1605 ms. Trace:
I20260807 19:50:16.163889 8911 rpcz_store.cc:276] 0807 19:50:14.544696 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.152922 (+608226us) service_pool.cc:224] Handling call
0807 19:50:15.153255 (+ 333us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.305754 (+152499us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.149998 (+844244us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b540016f3d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:16.150212 (+ 214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171356416}
I20260807 19:50:16.098436 13642 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:16.097023 13680 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:16.149484 13344 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:16.196357 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:16.183485 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
I20260807 19:50:16.100234 13577 consensus_queue.cc:1048] T 915c924a58ad4f3cbf22affe06485a15 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260807 19:50:16.230631 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:16.134439 13519 consensus_queue.cc:1048] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.051s
W20260807 19:50:16.233907 13700 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
W20260807 19:50:16.234689 13690 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.233s user 0.000s sys 0.001s
W20260807 19:50:16.235503 13681 log.cc:927] Time spent T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.363s user 0.001s sys 0.000s
I20260807 19:50:16.235958 13717 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:16.236526 13697 raft_consensus.cc:3055] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:50:16.183041 13695 raft_consensus.cc:493] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.134598 13598 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:16.147689 13216 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:16.271744 13645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s user 0.000s sys 0.004s
W20260807 19:50:16.272209 13645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s user 0.000s sys 0.004s
W20260807 19:50:16.273528 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 364882us
I20260807 19:50:16.247174 13629 raft_consensus.cc:697] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:16.184316 13344 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.260212 13627 raft_consensus.cc:3055] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Stepping down as leader of term 8
W20260807 19:50:16.240947 13241 consensus_meta.cc:287] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.098s user 0.021s sys 0.000s
I20260807 19:50:16.150533 13604 raft_consensus.cc:515] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:16.172559 13661 raft_consensus.cc:515] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.172681 13059 raft_consensus.cc:515] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.308238 13588 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:16.308750 13588 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:16.309213 13588 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:16.309643 13504 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.309969 13216 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:16.310232 13059 leader_election.cc:290] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:16.311609 13706 log.cc:927] Time spent T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.074s user 0.000s sys 0.000s
I20260807 19:50:16.313434 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:50:16.313787 13202 consensus_meta.cc:287] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.004s user 0.006s sys 0.006s
I20260807 19:50:16.314761 13645 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.315280 13645 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.305107 13543 raft_consensus.cc:3055] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:16.316736 13543 raft_consensus.cc:740] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:16.317095 13652 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.317633 13543 consensus_queue.cc:260] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.318795 13543 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:16.319242 13344 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.265956 13717 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.322105 13717 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.308668 13516 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:16.323011 13629 consensus_queue.cc:237] T 9d0dfd76f4dc484499bc12ddf5871c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.325002 13604 leader_election.cc:290] T bf699a9379e64939a7048af91aef93ec P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:16.274359 8740 leader_election.cc:304] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:16.269976 13695 raft_consensus.cc:515] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:16.247457 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 104896us
I20260807 19:50:16.302811 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:16.261320 13697 raft_consensus.cc:740] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:16.255795 13710 consensus_queue.cc:1059] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260807 19:50:16.230199 13692 consensus_queue.cc:1059] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260807 19:50:16.307210 8902 raft_consensus.cc:1240] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 3. Current term is 4. Ops: []
I20260807 19:50:16.307914 13702 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.212s
I20260807 19:50:16.328471 9023 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:16.224280 13680 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:16.184386 13593 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.330412 9850 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:16.335273 13661 leader_election.cc:290] T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:16.299820 13647 consensus_meta.cc:287] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.164s user 0.012s sys 0.004s
I20260807 19:50:16.338860 8904 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1776) took 1785 ms. Trace:
I20260807 19:50:16.423513 8904 rpcz_store.cc:276] 0807 19:50:14.553051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.495989 (+942938us) service_pool.cc:224] Handling call
0807 19:50:15.496250 (+ 261us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:15.880494 (+384244us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.338581 (+458087us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54003601d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:16.338779 (+ 198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832470400}
I20260807 19:50:16.327960 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:16.424961 8775 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Advancing to term 11
W20260807 19:50:16.308624 13687 log.cc:927] Time spent T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.364s user 0.000s sys 0.001s
W20260807 19:50:16.360400 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 86124.6us
I20260807 19:50:16.426467 8740 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:16.367954 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:16.406116 9021 raft_consensus.cc:1240] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260807 19:50:16.407234 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 325889us
I20260807 19:50:16.414060 13642 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.301028 13627 raft_consensus.cc:740] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:50:16.549748 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 282463us
I20260807 19:50:16.354820 13216 raft_consensus.cc:493] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
W20260807 19:50:16.466399 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:16.580003 8788 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.718s user 0.004s sys 0.012s
W20260807 19:50:16.569808 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 87296.4us
W20260807 19:50:16.580401 8788 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.718s user 0.005s sys 0.012s
I20260807 19:50:16.405083 13703 raft_consensus.cc:697] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:16.656030 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:16.780359 8775 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
W20260807 19:50:16.816599 8856 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s user 0.000s sys 0.027s
I20260807 19:50:16.589743 8779 raft_consensus.cc:1240] T 7abc25df9cc247e8b0354c9758c11208 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260807 19:50:16.801146 8780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s user 0.013s sys 0.022s
I20260807 19:50:16.817556 8972 leader_election.cc:304] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:50:16.808420 8854 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s user 0.003s sys 0.023s
W20260807 19:50:16.819454 8854 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s user 0.003s sys 0.024s
W20260807 19:50:16.822557 8856 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s user 0.000s sys 0.027s
W20260807 19:50:16.818430 8780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s user 0.014s sys 0.023s
I20260807 19:50:16.770905 8787 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: []
W20260807 19:50:16.837822 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 411419us
I20260807 19:50:16.336529 13697 consensus_queue.cc:260] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.844269 9017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57884 (request call id 1318) took 1529 ms. Trace:
I20260807 19:50:16.844779 8740 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:16.859612 9017 rpcz_store.cc:276] 0807 19:50:15.314729 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.314992 (+ 263us) service_pool.cc:224] Handling call
0807 19:50:15.324027 (+ 9035us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:15.324459 (+ 432us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:15.324490 (+ 31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:15.340753 (+ 16263us) log.cc:844] Serialized 26 byte log entry
0807 19:50:15.636626 (+295873us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:15.636667 (+ 41us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:15.636742 (+ 75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:16.844002 (+1207260us) raft_consensus.cc:1685] finished
0807 19:50:16.844030 (+ 28us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:16.844193 (+ 163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1738,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9879,"thread_start_us":241452,"threads_started":1,"wal-append.queue_time_us":323173}
W20260807 19:50:16.549963 13696 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.516s user 0.000s sys 0.002s
W20260807 19:50:16.868557 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 500698us
W20260807 19:50:16.800506 13560 consensus_meta.cc:287] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 1.875s user 0.012s sys 0.003s
I20260807 19:50:16.567088 13344 raft_consensus.cc:2749] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:16.673995 13555 raft_consensus.cc:3055] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
W20260807 19:50:16.799115 13712 log.cc:927] Time spent T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.250s user 0.000s sys 0.000s
I20260807 19:50:16.806357 13237 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.866338 13626 raft_consensus.cc:493] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.953557 13680 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:16.940828 13719 raft_consensus.cc:493] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:16.954320 13719 raft_consensus.cc:515] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.957043 13747 raft_consensus.cc:493] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.957610 13747 raft_consensus.cc:515] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.953470 13560 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801) with higher term. Message: Invalid argument: T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7.
I20260807 19:50:16.961297 13560 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.961798 13560 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.963526 8901 raft_consensus.cc:1240] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 7. Ops: []
W20260807 19:50:16.964042 13543 consensus_meta.cc:287] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.645s user 0.010s sys 0.000s
I20260807 19:50:16.966238 13626 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.435639 13729 raft_consensus.cc:2804] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:16.897156 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:16.901302 13716 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.100s user 0.000s sys 0.000s
I20260807 19:50:16.901435 13647 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:16.906412 13642 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.908416 13717 raft_consensus.cc:493] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:16.911175 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:16.911547 13692 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:16.912161 13728 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:16.913290 13718 raft_consensus.cc:2804] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:16.914041 13703 consensus_queue.cc:237] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:16.916463 13516 consensus_queue.cc:1059] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 19:50:16.917776 13598 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:16.920866 13697 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:16.578554 13480 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.599944 13519 raft_consensus.cc:3055] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:17.081524 13519 raft_consensus.cc:740] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:17.082654 13519 consensus_queue.cc:260] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.084187 13519 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:17.086685 13626 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.982401 8901 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1755) took 2436 ms. Trace:
I20260807 19:50:16.946758 13695 leader_election.cc:290] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:16.981135 13737 raft_consensus.cc:493] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.087777 8776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2166) took 1507 ms. Trace:
I20260807 19:50:16.949105 13733 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:16.965018 13237 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.089010 8776 rpcz_store.cc:276] 0807 19:50:15.580541 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.866891 (+286350us) service_pool.cc:224] Handling call
0807 19:50:15.900384 (+ 33493us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:15.911015 (+ 10631us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:15.911047 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:15.926385 (+ 15338us) log.cc:844] Serialized 26 byte log entry
0807 19:50:16.050656 (+124271us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:50:16.050704 (+ 48us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.050786 (+ 82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:17.087544 (+1036758us) raft_consensus.cc:1685] finished
0807 19:50:17.087568 (+ 24us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:17.087717 (+ 149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2756,"raft.queue_time_us":3291,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":67112,"thread_start_us":46719,"threads_started":1,"wal-append.queue_time_us":302660}
I20260807 19:50:17.087759 8901 rpcz_store.cc:276] 0807 19:50:14.546008 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.184223 (+638215us) service_pool.cc:224] Handling call
0807 19:50:15.191824 (+ 7601us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:16.963964 (+1772140us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.982063 (+ 18099us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540035f7d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:16.982294 (+ 231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4074197888}
W20260807 19:50:16.947981 12943 consensus_meta.cc:287] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.790s user 0.012s sys 0.003s
I20260807 19:50:17.091593 13733 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.092026 12943 raft_consensus.cc:515] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:17.073184 13588 consensus_meta.cc:287] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Time spent flushing consensus metadata: real 0.763s user 0.008s sys 0.001s
I20260807 19:50:17.094210 13733 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:16.891495 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 46818.4us
I20260807 19:50:16.980686 13555 raft_consensus.cc:740] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:16.629086 13216 raft_consensus.cc:515] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:16.984915 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 111907us
W20260807 19:50:16.965781 12927 consensus_meta.cc:287] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.377s user 0.012s sys 0.002s
I20260807 19:50:16.600406 13603 raft_consensus.cc:2749] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260807 19:50:16.977602 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 160144us
W20260807 19:50:16.948108 13086 consensus_meta.cc:287] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.100s user 0.009s sys 0.000s
I20260807 19:50:16.776023 13644 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:16.815140 13350 raft_consensus.cc:2749] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:17.028452 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:16.857846 13504 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.000356 13241 raft_consensus.cc:3060] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:16.875133 13652 raft_consensus.cc:493] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:17.027055 13404 consensus_meta.cc:287] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 1.046s user 0.012s sys 0.000s
I20260807 19:50:17.069676 9017 raft_consensus.cc:1240] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10. Ops: []
W20260807 19:50:17.073206 9850 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s user 0.000s sys 0.012s
W20260807 19:50:17.180302 9850 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.760s user 0.001s sys 0.013s
W20260807 19:50:17.181684 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 851349us
W20260807 19:50:17.073567 9848 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.021s sys 0.036s
W20260807 19:50:17.065110 13696 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.127s user 0.001s sys 0.000s
I20260807 19:50:17.016695 13647 raft_consensus.cc:493] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:17.016987 13729 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:17.069304 13629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s user 0.093s sys 0.146s
I20260807 19:50:17.088215 13737 raft_consensus.cc:515] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:16.951781 13549 consensus_meta.cc:287] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.532s user 0.014s sys 0.001s
I20260807 19:50:17.091115 13237 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.168627 13747 leader_election.cc:290] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:16.945662 13474 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:16.953058 13744 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.181676 8740 leader_election.cc:304] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.088392 13719 leader_election.cc:290] T f02f58b5ca51497592868892f144492a P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:17.243109 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 61524.4us
I20260807 19:50:17.243963 8740 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:17.097386 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:17.221753 8736 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s user 0.014s sys 0.035s
I20260807 19:50:17.184620 13404 raft_consensus.cc:2749] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:50:17.185108 9848 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s user 0.021s sys 0.036s
I20260807 19:50:17.062247 13717 raft_consensus.cc:515] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.227871 8968 leader_election.cc:304] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:17.178788 13448 consensus_meta.cc:287] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 1.529s user 0.009s sys 0.004s
I20260807 19:50:16.798851 13627 consensus_queue.cc:260] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.066011 13718 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.177026 9015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37380 (request call id 328) took 1265 ms. Trace:
I20260807 19:50:17.094372 12943 leader_election.cc:290] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:17.224139 13629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s user 0.093s sys 0.146s
I20260807 19:50:17.253662 8740 leader_election.cc:304] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.234513 13059 consensus_queue.cc:1048] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260807 19:50:17.265342 13237 raft_consensus.cc:2804] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260807 19:50:17.225519 13729 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:17.254361 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:17.094709 9020 raft_consensus.cc:1240] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:17.228861 13647 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:17.268594 9015 rpcz_store.cc:276] 0807 19:50:15.911294 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.911451 (+ 157us) service_pool.cc:224] Handling call
0807 19:50:15.911738 (+ 287us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:15.919204 (+ 7466us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:15.919240 (+ 36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:15.980379 (+ 61139us) log.cc:844] Serialized 26 byte log entry
0807 19:50:16.130730 (+150351us) raft_consensus.cc:1645] Marking committed up to 8
0807 19:50:16.130781 (+ 51us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.130880 (+ 99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:17.176762 (+1045882us) raft_consensus.cc:1685] finished
0807 19:50:17.176785 (+ 23us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:17.176952 (+ 167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7044,"raft.queue_time_us":158794,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10841,"thread_start_us":123179,"threads_started":1,"wal-append.queue_time_us":364612}
I20260807 19:50:17.113960 13626 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:17.273442 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 45649.9us
I20260807 19:50:17.221261 13344 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260807 19:50:17.268918 13753 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20260807 19:50:17.184926 13086 raft_consensus.cc:515] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.188304 13504 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.221112 12927 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.224526 13644 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.304129 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:17.094517 13588 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.066743 13692 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.305371 8790 raft_consensus.cc:1240] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
W20260807 19:50:17.316619 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 63021.9us
I20260807 19:50:17.268764 13718 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:17.265339 13577 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.318435 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:17.237075 13598 consensus_queue.cc:1059] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260807 19:50:17.188010 13652 raft_consensus.cc:515] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.239115 13731 consensus_queue.cc:1059] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260807 19:50:17.271729 13237 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.280570 13733 raft_consensus.cc:493] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:17.238513 13549 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.327780 9890 raft_consensus.cc:1240] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 3. Current term is 4. Ops: []
W20260807 19:50:17.332367 13739 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.162s user 0.001s sys 0.000s
I20260807 19:50:17.070966 13728 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.282459 13702 raft_consensus.cc:2804] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:17.335242 8740 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.333840 9017 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:17.303529 13719 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.265861 13762 raft_consensus.cc:2804] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:17.315992 13629 raft_consensus.cc:2804] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:17.347936 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:17.350723 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
W20260807 19:50:17.358125 13770 log.cc:927] Time spent T 00000000000000000000000000000000 P 9ee08994477a4e94a231c734242ac45f: Append to log took a long time: real 0.086s user 0.000s sys 0.001s
I20260807 19:50:17.363232 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:17.324252 13626 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } attrs { replace: true } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:17.369488 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 97442.5us
I20260807 19:50:17.340682 13771 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.327674 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf699a9379e64939a7048af91aef93ec" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:17.268966 13555 consensus_queue.cc:260] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.268096 13448 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.400264 13588 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.349687 13577 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.363838 13237 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Advancing to term 12
I20260807 19:50:17.406267 12927 leader_election.cc:290] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:17.406944 13344 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:17.253331 13744 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:17.525354 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 190226us
I20260807 19:50:17.379458 13733 raft_consensus.cc:515] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.410240 9015 raft_consensus.cc:2393] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 8: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:17.409024 9018 raft_consensus.cc:1240] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:17.525125 9892 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 5.
I20260807 19:50:17.500245 9017 raft_consensus.cc:2468] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:50:17.533318 8737 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:17.408736 13549 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:17.526769 8780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2167) took 1369 ms. Trace:
I20260807 19:50:17.528119 8740 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.533737 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f02f58b5ca51497592868892f144492a" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:17.538488 9019 raft_consensus.cc:1240] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:17.538499 8780 rpcz_store.cc:276] 0807 19:50:16.157526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.157675 (+ 149us) service_pool.cc:224] Handling call
0807 19:50:16.224941 (+ 67266us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:16.227640 (+ 2699us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:16.227670 (+ 30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:16.231891 (+ 4221us) log.cc:844] Serialized 26 byte log entry
0807 19:50:16.825844 (+593953us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:16.825891 (+ 47us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.825971 (+ 80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:17.526547 (+700576us) raft_consensus.cc:1685] finished
0807 19:50:17.526568 (+ 21us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:17.526703 (+ 135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65941,"raft.queue_time_us":99725,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"thread_start_us":562458,"threads_started":1,"wal-append.queue_time_us":825281}
W20260807 19:50:17.539495 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 173798us
I20260807 19:50:17.545849 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:17.546486 9017 raft_consensus.cc:2393] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 11: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:17.410532 13781 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.410372 13404 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.394366 13762 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.394672 13629 raft_consensus.cc:493] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.449728 13737 leader_election.cc:290] T e10d78176f8545489a4700b7bd08fec3 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.526300 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:17.549280 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe08dfdbfbdb4fd08342efd539590bf2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:17.548605 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0dd3f602c145219dc8717e182ff568" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:17.568583 9015 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:17.569757 9015 raft_consensus.cc:1240] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 3. Current term is 4. Ops: []
I20260807 19:50:17.572990 9890 raft_consensus.cc:2393] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 4: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:50:17.575014 13577 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:17.589324 13519 consensus_meta.cc:287] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.505s user 0.005s sys 0.005s
I20260807 19:50:17.538268 13555 raft_consensus.cc:3060] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:17.632145 13598 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:17.656023 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 114443us
I20260807 19:50:17.656908 8968 leader_election.cc:304] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:17.568522 13629 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:17.411464 13719 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.557096 13736 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:17.567210 13216 leader_election.cc:290] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:17.660102 8853 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 216701us
I20260807 19:50:17.530858 13344 raft_consensus.cc:515] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.631870 13737 raft_consensus.cc:3055] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:17.566749 13762 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:17.672645 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:17.689028 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 155811us
I20260807 19:50:17.691295 8856 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:17.527137 13771 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.564783 13404 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.657774 13241 raft_consensus.cc:2749] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:17.657523 13602 consensus_queue.cc:1059] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260807 19:50:17.697937 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 169899us
I20260807 19:50:17.688489 13549 raft_consensus.cc:2804] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader election won for term 11
I20260807 19:50:17.693588 13086 leader_election.cc:290] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:17.669046 13786 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:17.718003 8740 leader_election.cc:304] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.593653 13781 raft_consensus.cc:515] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.664463 13647 raft_consensus.cc:515] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.661471 13598 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.699661 13790 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.708658 13519 raft_consensus.cc:493] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:17.703365 13737 raft_consensus.cc:740] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:17.720255 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:17.711359 13644 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.756395 13717 leader_election.cc:290] T 65e8bf81123c4245acf858d947af51c1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.706300 13791 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:17.728890 13692 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:17.763880 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:17.786113 13793 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:17.786681 13793 raft_consensus.cc:493] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.787263 13793 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:17.760015 13549 raft_consensus.cc:697] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:17.728493 13786 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.701042 13736 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.761699 13802 raft_consensus.cc:493] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.770437 13448 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:17.784708 13718 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.754732 13790 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.761307 13702 raft_consensus.cc:697] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:17.758528 13519 raft_consensus.cc:515] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.799743 13795 raft_consensus.cc:2764] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260807 19:50:17.788662 13404 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:17.817539 13729 consensus_meta.cc:287] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.511s user 0.013s sys 0.000s
W20260807 19:50:17.819914 13777 log.cc:927] Time spent T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.278s user 0.000s sys 0.001s
I20260807 19:50:17.812666 13733 leader_election.cc:290] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.802999 13786 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:17.764526 13791 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.816023 13744 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.846364 13812 raft_consensus.cc:3055] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Stepping down as leader of term 3
W20260807 19:50:17.836184 13474 consensus_meta.cc:287] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.595s user 0.009s sys 0.000s
I20260807 19:50:17.806244 13728 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:17.824828 13602 raft_consensus.cc:2804] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260807 19:50:17.847149 13549 consensus_queue.cc:237] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.852530 13543 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:17.853181 13543 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.836611 13404 raft_consensus.cc:2804] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:50:17.856256 13404 raft_consensus.cc:493] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:17.856909 13404 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:17.801563 13737 consensus_queue.cc:260] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.868534 13602 raft_consensus.cc:697] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:17.869714 13602 consensus_queue.cc:237] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:17.870527 13719 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.837134 13729 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.858880 13626 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:17.863916 13812 raft_consensus.cc:740] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:17.821389 13802 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:17.880712 13737 raft_consensus.cc:3060] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:50:17.888734 13792 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.093s user 0.001s sys 0.000s
W20260807 19:50:17.892446 13799 log.cc:927] Time spent T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.069s user 0.000s sys 0.001s
W20260807 19:50:17.819569 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 101670us
I20260807 19:50:17.933360 8740 leader_election.cc:304] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.937592 8740 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:17.941001 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:17.941987 8780 raft_consensus.cc:1240] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:17.942546 8775 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:17.945266 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:17.946303 8780 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:17.953464 8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1586) took 2451 ms. Trace:
W20260807 19:50:17.962492 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 271163us
I20260807 19:50:17.954130 8788 rpcz_store.cc:276] 0807 19:50:15.502071 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.542628 (+ 40557us) service_pool.cc:224] Handling call
0807 19:50:15.542953 (+ 325us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:15.543361 (+ 408us) raft_consensus.cc:1534] Early marking committed up to index 4
0807 19:50:15.543710 (+ 349us) log.cc:844] Serialized 14 byte log entry
0807 19:50:15.860951 (+317241us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:15.861639 (+ 688us) log.cc:844] Serialized 26 byte log entry
0807 19:50:15.861803 (+ 164us) raft_consensus.cc:1645] Marking committed up to 5
0807 19:50:16.586523 (+724720us) log.cc:844] Serialized 14 byte log entry
0807 19:50:16.586884 (+ 361us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:16.586968 (+ 84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:17.832809 (+1245841us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b18003d2e20. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f6881419f31 00007f68814266d1 00007f68784fdd33 00007f6878500121 00007f68785000d9 00007f6878500069 00007f6878500031 00007f68784ff32d 00000000003aa554 00000000003aa399 00007f68784e11ea
0807 19:50:17.952784 (+119975us) raft_consensus.cc:1685] finished
0807 19:50:17.952831 (+ 47us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:17.953155 (+ 324us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30404,"prepare.queue_time_us":23988,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":260,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":29531,"spinlock_wait_cycles":14947584,"thread_start_us":1041093,"threads_started":2,"wal-append.queue_time_us":348216}
I20260807 19:50:17.969754 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
W20260807 19:50:17.973454 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 141011us
I20260807 19:50:17.974781 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:17.976085 9890 raft_consensus.cc:1813] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Handling vote request from an unknown peer 671d8754170646eaac5748745eba58b4
I20260807 19:50:17.977077 9890 raft_consensus.cc:2410] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260807 19:50:17.985318 8775 raft_consensus.cc:2468] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:17.987589 13791 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:17.989516 13802 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.993700 13344 leader_election.cc:290] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.994837 13718 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:17.888911 13779 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.126s user 0.000s sys 0.001s
I20260807 19:50:17.889158 13771 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.004125 13795 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:18.008744 13644 raft_consensus.cc:493] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:18.012202 8788 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:18.005285 13795 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.017272 13781 leader_election.cc:290] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:17.850185 13702 consensus_queue.cc:237] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.024823 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.030184 13814 raft_consensus.cc:493] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
I20260807 19:50:18.030665 8780 raft_consensus.cc:2468] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
W20260807 19:50:18.031733 13697 consensus_meta.cc:287] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.917s user 0.013s sys 0.001s
I20260807 19:50:18.033228 13598 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.020812 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:18.034655 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:17.893548 13812 consensus_queue.cc:260] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.009054 13801 raft_consensus.cc:2749] T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260807 19:50:18.048820 8775 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:18.053138 13560 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.031527 13733 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.031224 13814 raft_consensus.cc:515] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.034708 13762 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.066056 13647 leader_election.cc:290] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.066390 13613 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.058357 13519 leader_election.cc:290] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.071547 13816 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260807 19:50:18.074599 13801 raft_consensus.cc:3055] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:18.075053 13801 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:18.067198 13805 consensus_queue.cc:1059] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260807 19:50:18.062902 13791 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:18.091058 8775 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:18.070294 13644 raft_consensus.cc:515] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.077589 8906 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:18.082309 8915 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54662 (request call id 1308) took 2291 ms (client timeout 2143 ms). Trace:
I20260807 19:50:18.097669 9885 raft_consensus.cc:1240] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 10 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:18.098052 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:18.100869 13791 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:18.101367 9023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1752) took 2455 ms. Trace:
I20260807 19:50:18.103313 8910 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1785) took 2121 ms. Trace:
W20260807 19:50:18.102420 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 165042us
I20260807 19:50:18.105528 9008 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1758) took 1804 ms. Trace:
I20260807 19:50:18.106444 13602 raft_consensus.cc:493] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.107252 13602 raft_consensus.cc:515] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.107740 9018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1755) took 2245 ms. Trace:
I20260807 19:50:18.108675 13683 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:18.109022 8740 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:18.101787 9023 rpcz_store.cc:276] 0807 19:50:15.645707 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.645847 (+ 140us) service_pool.cc:224] Handling call
0807 19:50:15.651763 (+ 5916us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:16.405535 (+753772us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.100779 (+1695244us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b540006bdd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.101114 (+ 335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555733376}
I20260807 19:50:18.106053 9008 rpcz_store.cc:276] 0807 19:50:16.301517 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.301641 (+ 124us) service_pool.cc:224] Handling call
0807 19:50:16.301955 (+ 314us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:16.580144 (+278189us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:16.580176 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:16.580218 (+ 42us) raft_consensus.cc:1645] Marking committed up to 6
0807 19:50:16.580250 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.105011 (+1524761us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400449654. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.105082 (+ 71us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:18.105328 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639234944}
I20260807 19:50:18.076452 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:18.097988 8906 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54668 (request call id 1773) took 3545 ms. Trace:
I20260807 19:50:18.108420 9018 rpcz_store.cc:276] 0807 19:50:15.862198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.862328 (+ 130us) service_pool.cc:224] Handling call
0807 19:50:15.862654 (+ 326us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:17.538607 (+1675953us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.107188 (+568581us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540002d854. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.107539 (+ 351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3555901184}
I20260807 19:50:18.116844 9850 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:18.106355 13560 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.103771 8910 rpcz_store.cc:276] 0807 19:50:15.982075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.080821 (+ 98746us) service_pool.cc:224] Handling call
0807 19:50:18.094475 (+2013654us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:18.095091 (+ 616us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:18.095123 (+ 32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0807 19:50:18.101594 (+ 6471us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:18.101696 (+ 102us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.101801 (+ 105us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:18.102862 (+ 1061us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b540016e9d0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b603c8d 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:18.103064 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4605021440}
I20260807 19:50:18.076161 13729 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.120240 8788 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
W20260807 19:50:18.097020 8915 rpcz_store.cc:269] 0807 19:50:15.790850 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.791014 (+ 164us) service_pool.cc:224] Handling call
0807 19:50:18.081722 (+2290708us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540016e9d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:18.081968 (+ 246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360815488}
I20260807 19:50:18.122730 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.114769 8906 rpcz_store.cc:276] 0807 19:50:14.552267 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.478520 (+926253us) service_pool.cc:224] Handling call
0807 19:50:17.180154 (+1701634us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4c000008c0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f68798cdfc0 00007f68798defa9 00007f68798df68f 00007f68798a2e7b 00007f68798d14dc 00007f68798d199e 00007f6879b619eb 00007f687b560875 00007f687b5b16ef 00007f687b5fbe35 00007f68805fdae0
0807 19:50:18.077148 (+896994us) spinlock_profiling.cc:243] Waited 995 us on lock 0x7b4c000094c0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f68798cdfc0 00007f68798defa9 00007f68798df68f 00007f68798a2e7b 00007f68798d14dc 00007f68798d199e 00007f6879b619eb 00007f687b560875 00007f687b5b16ef 00007f687b5fbe35 00007f68805fdae0
0807 19:50:18.097699 (+ 20551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7452288}
I20260807 19:50:18.125427 13819 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.128808 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:18.126766 13771 consensus_queue.cc:1059] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260807 19:50:18.133067 9019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1757) took 1832 ms. Trace:
I20260807 19:50:18.082953 13790 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.133540 9019 rpcz_store.cc:276] 0807 19:50:16.301112 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.301261 (+ 149us) service_pool.cc:224] Handling call
0807 19:50:16.301659 (+ 398us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:17.539642 (+1237983us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.113018 (+573376us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54000d57d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.132868 (+ 19850us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2844029312}
I20260807 19:50:18.109589 13683 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.137846 9890 raft_consensus.cc:2393] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 6: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:50:18.113245 13697 raft_consensus.cc:493] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:18.140609 13697 raft_consensus.cc:515] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.143144 13394 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.151669 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.153230 13814 leader_election.cc:290] T 10901e14a5e8471291ad1d7c203b6740 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.127300 13795 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.154254 9020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1760) took 1107 ms. Trace:
W20260807 19:50:18.156811 9838 leader_election.cc:336] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.859s (SENT)
I20260807 19:50:18.156224 13560 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.158397 13812 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:18.159444 13762 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.160578 13644 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.080816 13733 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.168497 9838 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:18.134505 13771 raft_consensus.cc:3055] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:18.167936 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.196146 8780 raft_consensus.cc:2410] T f4ace421d3564015bfd7eb6cff4b78fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260807 19:50:18.129186 13819 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.201047 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.202261 9017 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:50:18.202790 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:18.203837 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 63750.8us
I20260807 19:50:18.167877 9020 rpcz_store.cc:276] 0807 19:50:17.027046 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:17.027561 (+ 515us) service_pool.cc:224] Handling call
0807 19:50:17.027918 (+ 357us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:17.277241 (+249323us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.121547 (+844306us) spinlock_profiling.cc:243] Waited 66.6 ms on lock 0x7b5400190fd4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.134524 (+ 12977us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153126656}
I20260807 19:50:18.170059 13802 consensus_queue.cc:1059] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260807 19:50:18.206987 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 98072.4us
I20260807 19:50:18.207650 13719 raft_consensus.cc:3055] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:18.208083 8740 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
I20260807 19:50:18.209794 8740 leader_election.cc:304] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:18.211946 13719 raft_consensus.cc:740] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:18.212641 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.197278 13771 raft_consensus.cc:740] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:18.224761 8775 raft_consensus.cc:2410] T 2d69c395b87b43c0bf7c3d1efb63e699 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260807 19:50:18.169732 13626 consensus_queue.cc:1059] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260807 19:50:18.077680 13801 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.169593 13608 consensus_queue.cc:1059] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260807 19:50:18.170171 13555 consensus_queue.cc:1059] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260807 19:50:18.175521 13791 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.222033 13697 leader_election.cc:290] T 138f0e8a31994c549dd743c9ba49d6ee P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.234630 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.240448 13762 raft_consensus.cc:2749] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:18.221349 13795 raft_consensus.cc:2764] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 LEADER]: Leader pre-election decision vote started in defunct term 9: won
I20260807 19:50:18.245412 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.244735 13615 raft_consensus.cc:2749] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260807 19:50:18.249747 8775 raft_consensus.cc:1240] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 7. Current term is 8. Ops: []
I20260807 19:50:18.232899 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:18.261866 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.253677 13626 consensus_queue.cc:1243] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer a38f4d6088cf4c8cb510200ca0555801 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260807 19:50:18.268672 13819 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:18.273880 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 64294.9us
I20260807 19:50:18.274787 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.275872 9027 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 10.
I20260807 19:50:18.278024 9019 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 9.
I20260807 19:50:18.278992 8787 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2168) took 2121 ms. Trace:
I20260807 19:50:18.279402 8787 rpcz_store.cc:276] 0807 19:50:16.157857 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.158261 (+ 404us) service_pool.cc:224] Handling call
0807 19:50:16.158712 (+ 451us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:16.830659 (+671947us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.278507 (+1447848us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b5400328554. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.278805 (+ 298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407360768}
W20260807 19:50:18.280472 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.281615 8789 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2162) took 2826 ms. Trace:
W20260807 19:50:18.283795 13828 log.cc:927] Time spent T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.133s user 0.000s sys 0.001s
W20260807 19:50:18.284528 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.264741 13835 raft_consensus.cc:2749] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260807 19:50:18.285673 13771 consensus_queue.cc:260] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.287009 13771 raft_consensus.cc:3060] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:50:18.284711 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 128190us
W20260807 19:50:18.269599 13738 log.cc:927] Time spent T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 1.454s user 0.008s sys 0.008s
I20260807 19:50:18.282074 8789 rpcz_store.cc:276] 0807 19:50:15.454991 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.455139 (+ 148us) service_pool.cc:224] Handling call
0807 19:50:15.455528 (+ 389us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:15.580120 (+124592us) raft_consensus.cc:1534] Early marking committed up to index 3
0807 19:50:15.626831 (+ 46711us) log.cc:844] Serialized 14 byte log entry
0807 19:50:15.872336 (+245505us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:15.891989 (+ 19653us) log.cc:844] Serialized 26 byte log entry
0807 19:50:15.892144 (+ 155us) raft_consensus.cc:1645] Marking committed up to 3
0807 19:50:15.892176 (+ 32us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.280727 (+2388551us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54001603d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.280902 (+ 175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:18.281002 (+ 100us) raft_consensus.cc:1685] finished
0807 19:50:18.281129 (+ 127us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:18.281400 (+ 271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10699,"prepare.queue_time_us":54252,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":14562,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":18606,"spinlock_wait_cycles":285613952,"thread_start_us":209809,"threads_started":1,"wal-append.queue_time_us":274702}
I20260807 19:50:18.299341 13683 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.301085 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.306934 13560 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.272451 13719 consensus_queue.cc:260] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.311149 8902 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53310 (request call id 434) took 3751 ms. Trace:
I20260807 19:50:18.311626 8902 rpcz_store.cc:276] 0807 19:50:14.560082 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.676180 (+1116098us) service_pool.cc:224] Handling call
0807 19:50:15.686708 (+ 10528us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:16.405529 (+718821us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.310612 (+1905083us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001ed7d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.310964 (+ 352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426604928}
W20260807 19:50:18.319792 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.309032 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:18.305373 13602 leader_election.cc:290] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.334386 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:18.327349 13801 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:18.344565 13786 raft_consensus.cc:515] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.347362 9885 raft_consensus.cc:2393] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 10: Already voted for candidate 39975243456d4452b4a4870ac7ebafc5 in this term.
I20260807 19:50:18.347635 9021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1754) took 2540 ms. Trace:
I20260807 19:50:18.351743 9888 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:18.352581 13786 leader_election.cc:290] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.351927 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.350859 9021 rpcz_store.cc:276] 0807 19:50:15.806720 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.806888 (+ 168us) service_pool.cc:224] Handling call
0807 19:50:15.807240 (+ 352us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:16.437424 (+630184us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.345126 (+1907702us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540034dad4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.347440 (+ 2314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376538496}
I20260807 19:50:18.338521 13733 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.356710 13615 consensus_queue.cc:1059] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260807 19:50:18.380208 9019 raft_consensus.cc:2468] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 8.
I20260807 19:50:18.380441 9020 raft_consensus.cc:1275] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260807 19:50:18.360224 13565 raft_consensus.cc:3060] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Advancing to term 8
W20260807 19:50:18.400610 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.380218 9888 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:18.395720 9892 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
W20260807 19:50:18.401052 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.362905 13404 raft_consensus.cc:515] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.395921 8740 leader_election.cc:304] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:18.394459 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
W20260807 19:50:18.413988 13793 consensus_meta.cc:287] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.582s user 0.010s sys 0.011s
I20260807 19:50:18.414677 13793 raft_consensus.cc:515] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.424031 8775 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 10. Current term is 11. Ops: []
I20260807 19:50:18.416949 13394 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:18.425257 13629 consensus_meta.cc:287] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.677s user 0.017s sys 0.001s
I20260807 19:50:18.430676 13786 raft_consensus.cc:2804] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:50:18.430720 13394 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.434226 13394 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:18.431327 13786 raft_consensus.cc:493] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:18.435621 13786 raft_consensus.cc:3060] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:18.438874 8970 leader_election.cc:304] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:18.440366 13516 raft_consensus.cc:2804] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:18.440804 13516 raft_consensus.cc:493] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:18.441300 13516 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:18.441036 13474 consensus_queue.cc:1059] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260807 19:50:18.439178 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 43495.9us
I20260807 19:50:18.420416 13736 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:18.456287 13793 leader_election.cc:290] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:18.458480 8739 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.458775 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:18.452739 13728 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:18.454094 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 1.22537e+06us
I20260807 19:50:18.460915 8972 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:18.452696 13629 raft_consensus.cc:515] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.463927 13629 leader_election.cc:290] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:18.454751 13836 log.cc:927] Time spent T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.157s user 0.001s sys 0.000s
I20260807 19:50:18.464605 13404 leader_election.cc:290] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 11 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:18.454298 13237 consensus_meta.cc:287] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.923s user 0.016s sys 0.006s
I20260807 19:50:18.466495 8913 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54668 (request call id 1787) took 2483 ms. Trace:
I20260807 19:50:18.459496 9885 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:18.454186 8740 leader_election.cc:304] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:50:18.474805 8849 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.893s (SENT) [suppressed 30 similar messages]
W20260807 19:50:18.475850 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.476778 13728 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.478444 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e10d78176f8545489a4700b7bd08fec3" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.478631 13736 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.479471 13728 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:18.479296 13736 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.476897 9886 raft_consensus.cc:1240] T be133eef91634be592094b95357ee200 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:18.482707 13666 raft_consensus.cc:3055] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:18.483193 13666 raft_consensus.cc:740] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:18.484303 13666 consensus_queue.cc:260] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.487463 13788 raft_consensus.cc:2804] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:18.486594 13504 leader_election.cc:290] T 4ec2edf1da424177aaad8b93086118b1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.471637 8913 rpcz_store.cc:276] 0807 19:50:15.982605 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.146545 (+163940us) service_pool.cc:224] Handling call
0807 19:50:16.147023 (+ 478us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:16.356493 (+209470us) raft_consensus.cc:1534] Early marking committed up to index 0
0807 19:50:16.356526 (+ 33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0807 19:50:16.357564 (+ 1038us) log.cc:844] Serialized 26 byte log entry
0807 19:50:16.854982 (+497418us) raft_consensus.cc:1645] Marking committed up to 4
0807 19:50:16.855026 (+ 44us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.465729 (+1610703us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x7b54002fc954. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.465812 (+ 83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0807 19:50:18.465894 (+ 82us) raft_consensus.cc:1685] finished
0807 19:50:18.465909 (+ 15us) raft_consensus.cc:1691] UpdateReplicas() finished
0807 19:50:18.466279 (+ 370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7183,"raft.queue_time_us":81433,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"spinlock_wait_cycles":87394688,"thread_start_us":399640,"threads_started":2,"wal-append.queue_time_us":554165,"wal-append.run_cpu_time_us":3385,"wal-append.run_wall_time_us":1232758}
I20260807 19:50:18.484778 8789 raft_consensus.cc:1240] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:18.490999 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.481359 13627 raft_consensus.cc:3060] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:18.491393 8972 leader_election.cc:304] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:18.488480 13202 raft_consensus.cc:3055] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:18.496188 13394 consensus_queue.cc:1059] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260807 19:50:18.497593 13394 raft_consensus.cc:2804] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260807 19:50:18.498015 13394 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:18.498476 13394 raft_consensus.cc:3060] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:18.498677 13645 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:18.499480 13645 raft_consensus.cc:2804] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:18.499806 13645 raft_consensus.cc:493] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:18.500123 13645 raft_consensus.cc:3060] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:18.500259 8789 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:18.470445 13771 raft_consensus.cc:493] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.501295 13202 raft_consensus.cc:740] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.488058 13788 raft_consensus.cc:493] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a38f4d6088cf4c8cb510200ca0555801)
W20260807 19:50:18.501782 13737 consensus_meta.cc:287] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.526s user 0.005s sys 0.010s
I20260807 19:50:18.481707 13736 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:18.488509 13666 raft_consensus.cc:3060] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:18.511617 9017 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 10.
I20260807 19:50:18.513252 13202 consensus_queue.cc:260] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.514923 8968 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:18.515625 13771 raft_consensus.cc:515] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:18.526257 13776 log.cc:927] Time spent T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 1.124s user 0.000s sys 0.003s
I20260807 19:50:18.530400 13786 raft_consensus.cc:515] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.508481 13788 raft_consensus.cc:3060] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:18.489864 13237 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.556183 13202 raft_consensus.cc:3060] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:18.556519 13660 raft_consensus.cc:3055] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:18.556974 13660 raft_consensus.cc:740] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.558089 13660 consensus_queue.cc:260] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.559573 13660 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:18.562335 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:18.563257 8913 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10.
W20260807 19:50:18.566747 13838 log.cc:927] Time spent T be133eef91634be592094b95357ee200 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.203s user 0.000s sys 0.002s
W20260807 19:50:18.568598 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 53871.7us
I20260807 19:50:18.569695 8968 leader_election.cc:304] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:18.570053 13695 raft_consensus.cc:2804] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:18.570709 13695 raft_consensus.cc:697] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.571223 13652 leader_election.cc:290] T 915c924a58ad4f3cbf22affe06485a15 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.573585 13771 leader_election.cc:290] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.566610 13692 consensus_queue.cc:1059] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260807 19:50:18.575691 13682 raft_consensus.cc:3055] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:18.576159 8779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51022 (request call id 1590) took 3073 ms. Trace:
I20260807 19:50:18.576217 13682 raft_consensus.cc:740] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.577057 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.577716 8788 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:18.580506 13682 consensus_queue.cc:260] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.581729 13682 raft_consensus.cc:3060] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:18.575923 9890 raft_consensus.cc:3055] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEADER]: Stepping down as leader of term 3
I20260807 19:50:18.586016 9890 raft_consensus.cc:740] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:18.576664 8779 rpcz_store.cc:276] 0807 19:50:15.503071 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:15.549847 (+ 46776us) service_pool.cc:224] Handling call
0807 19:50:15.550201 (+ 354us) raft_consensus.cc:1471] Updating replica for 1 ops
0807 19:50:16.842179 (+1291978us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.031861 (+1189682us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b54000b0ca0. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b57c1a0 00007f687b607de2 00007f687b6041b6 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22
0807 19:50:18.549916 (+518055us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540040b5d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.575958 (+ 26042us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2448422144}
I20260807 19:50:18.575666 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.582417 13736 raft_consensus.cc:2804] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:18.581632 8968 leader_election.cc:304] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:18.591123 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 58943.1us
I20260807 19:50:18.577417 13786 leader_election.cc:290] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 5 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.591109 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.598722 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6391ca8e3f6476d97c1983eda89cdd2" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.600610 8780 raft_consensus.cc:1240] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 12 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 11. Current term is 12. Ops: []
I20260807 19:50:18.603957 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.612460 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.611812 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.571828 13695 consensus_queue.cc:237] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.616297 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.628221 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:18.633997 13736 raft_consensus.cc:697] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.635269 13736 consensus_queue.cc:237] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.637341 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2fe2de08a4ba49eea927f371b4bf0e5f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.638113 8783 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:18.640657 9890 consensus_queue.cc:260] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.641798 9890 raft_consensus.cc:3060] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:18.642619 9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.647590 13543 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:18.648917 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.649936 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.605890 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab8e617d098d425cbddc5d78da43ab99" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:18.655298 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.655678 8778 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
W20260807 19:50:18.660997 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 59283.8us
I20260807 19:50:18.661453 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.661612 9019 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260807 19:50:18.656306 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.652305 8913 raft_consensus.cc:2368] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 6. Current term is 7.
I20260807 19:50:18.654497 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:18.664597 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.665779 9890 raft_consensus.cc:1275] T 92d07404635f47998c0f61e0e7f5b732 P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260807 19:50:18.652014 8788 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 6.
I20260807 19:50:18.667305 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:18.668483 9022 raft_consensus.cc:1240] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:18.668948 8910 raft_consensus.cc:1240] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [term 4 LEADER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 3. Current term is 4. Ops: []
I20260807 19:50:18.669888 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.675611 8915 raft_consensus.cc:2376] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 10. Re-sending same reply.
W20260807 19:50:18.669358 13588 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260807 19:50:18.682207 13642 raft_consensus.cc:493] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.682916 13642 raft_consensus.cc:515] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.686107 9885 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:18.688524 8968 leader_election.cc:304] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:18.689435 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 72179.9us
I20260807 19:50:18.691344 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2b65bfd5b3948cea9db5090e802132d" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.692126 8906 raft_consensus.cc:3060] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:18.655922 13728 raft_consensus.cc:2804] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:18.696930 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:18.697458 9021 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:50:18.682451 8779 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:18.652486 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.669884 13237 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 12 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:18.711246 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.714613 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb51072a064989a2056cfa1484f4a5" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.725210 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:18.731448 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.732640 8788 raft_consensus.cc:2393] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 39975243456d4452b4a4870ac7ebafc5 in current term 6: Already voted for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in this term.
I20260807 19:50:18.733721 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.735209 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:18.737799 8849 leader_election.cc:336] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.544s (SENT)
I20260807 19:50:18.738555 8849 leader_election.cc:304] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:18.751614 13647 consensus_queue.cc:1059] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260807 19:50:18.755152 13647 raft_consensus.cc:3055] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:18.755620 13823 raft_consensus.cc:3055] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:18.756223 13823 raft_consensus.cc:740] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:18.756510 13549 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.757230 13549 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:18.757623 13823 consensus_queue.cc:260] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.758869 13823 raft_consensus.cc:3060] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:18.724012 13728 raft_consensus.cc:697] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.790095 13728 consensus_queue.cc:237] T 479d60d809ec458397b1360e820ed150 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.781534 13692 consensus_queue.cc:1059] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
W20260807 19:50:18.783636 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:18.792465 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.784210 13723 raft_consensus.cc:2749] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260807 19:50:18.785054 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8160be756c34002a2ba00cf8deaa10a" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.722374 13613 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260807 19:50:18.759976 13102 consensus_queue.cc:1059] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
I20260807 19:50:18.808282 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.825671 12927 raft_consensus.cc:2804] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:18.760489 13647 raft_consensus.cc:740] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:18.832561 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.841903 8788 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:18.843631 8778 raft_consensus.cc:1275] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 19:50:18.827642 13102 raft_consensus.cc:3055] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Stepping down as leader of term 7
I20260807 19:50:18.853937 13598 raft_consensus.cc:493] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:18.854542 13598 raft_consensus.cc:515] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.836786 12927 raft_consensus.cc:697] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:18.858634 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 120909us
I20260807 19:50:18.852133 8968 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:18.846661 13102 raft_consensus.cc:740] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:18.861049 9850 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4
W20260807 19:50:18.868996 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.869470 8778 raft_consensus.cc:1240] T ab8e617d098d425cbddc5d78da43ab99 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 5 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 4. Current term is 5. Ops: []
I20260807 19:50:18.872236 8737 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792; no voters:
W20260807 19:50:18.873103 8737 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 44900.1us
I20260807 19:50:18.873430 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f72bb6e69514b33881da72833a29378" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:18.880110 9008 raft_consensus.cc:1240] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 3. Current term is 4. Ops: []
I20260807 19:50:18.880781 13692 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:50:18.881312 13692 raft_consensus.cc:493] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:18.881721 13692 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:18.882755 13237 raft_consensus.cc:2804] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader election won for term 6
I20260807 19:50:18.883401 13237 raft_consensus.cc:697] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 LEADER]: Becoming Leader. State: Replica: 8e9ad2b5e2a94d4faff90af735c6d792, State: Running, Role: LEADER
I20260807 19:50:18.884490 13237 consensus_queue.cc:237] T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:18.886494 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:18.887060 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:18.887387 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.889462 13102 consensus_queue.cc:260] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:18.890995 13102 raft_consensus.cc:3060] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:18.896974 8779 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 7.
I20260807 19:50:18.898097 8906 raft_consensus.cc:2468] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
I20260807 19:50:18.897972 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:18.908735 8902 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
W20260807 19:50:18.912145 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.912240 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:18.911049 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:18.917296 8783 raft_consensus.cc:2468] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:18.918188 8788 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
W20260807 19:50:18.923306 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 62388.4us
I20260807 19:50:18.910048 13615 raft_consensus.cc:2804] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:18.922987 13727 consensus_queue.cc:1059] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260807 19:50:18.933660 8783 raft_consensus.cc:1240] T 17510c7d48eb44ceb3d8c645ceaa28fc P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:18.936112 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d69c395b87b43c0bf7c3d1efb63e699" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:18.938205 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.943689 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:18.936165 9023 raft_consensus.cc:1240] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 8. Current term is 9. Ops: []
W20260807 19:50:18.948303 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:18.948999 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:18.928580 13549 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:18.954135 13202 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:18.938544 13819 consensus_queue.cc:1059] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260807 19:50:18.965214 8790 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2174) took 2736 ms. Trace:
I20260807 19:50:18.985179 8901 raft_consensus.cc:2468] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:18.979286 8968 leader_election.cc:304] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:18.966583 13719 raft_consensus.cc:3060] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
W20260807 19:50:18.967361 9838 leader_election.cc:336] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.006s (SENT)
I20260807 19:50:18.947016 13615 raft_consensus.cc:697] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [term 10 LEADER]: Becoming Leader. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:18.993844 8904 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260807 19:50:18.988461 8790 rpcz_store.cc:276] 0807 19:50:16.229019 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:16.672198 (+443179us) service_pool.cc:224] Handling call
0807 19:50:16.777084 (+104886us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:17.334220 (+557136us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.962918 (+1628698us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54000f4e54. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.965025 (+ 2107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214444800}
I20260807 19:50:18.969883 8775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51024 (request call id 2190) took 1040 ms. Trace:
I20260807 19:50:18.970141 8778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.988534 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.954952 8856 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:18.985013 13202 raft_consensus.cc:493] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:18.968894 8787 raft_consensus.cc:1240] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:18.998001 8778 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:19.002892 13615 consensus_queue.cc:237] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.004806 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:19.005497 8790 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:19.006565 8783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:18.997278 8775 rpcz_store.cc:276] 0807 19:50:17.929600 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:18.287775 (+358175us) service_pool.cc:224] Handling call
0807 19:50:18.297293 (+ 9518us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:18.424525 (+127232us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:18.969590 (+545065us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540040b0d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:18.969781 (+ 191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286438272}
I20260807 19:50:19.007288 8783 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:19.007318 8778 raft_consensus.cc:1275] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260807 19:50:19.008921 8778 pending_rounds.cc:85] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:50:19.009133 8778 pending_rounds.cc:107] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260807 19:50:19.009394 8778 raft_consensus.cc:2889] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:19.009394 8783 raft_consensus.cc:1240] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 6. Current term is 7. Ops: []
I20260807 19:50:19.010948 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.006620 8790 raft_consensus.cc:1240] T d0e0dcc92bf04b66a4b58d4e68b01703 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:18.979815 13519 raft_consensus.cc:3055] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Stepping down as leader of term 11
I20260807 19:50:19.017383 9838 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:19.011574 8903 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 4.
I20260807 19:50:19.019100 8902 raft_consensus.cc:2410] T 2d69c395b87b43c0bf7c3d1efb63e699 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260807 19:50:19.012534 13516 consensus_meta.cc:287] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.571s user 0.016s sys 0.018s
I20260807 19:50:19.018574 13394 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.035674 13685 raft_consensus.cc:3055] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:19.036257 13685 raft_consensus.cc:740] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:19.037382 13685 consensus_queue.cc:260] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.038890 13685 raft_consensus.cc:3060] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:19.039577 13642 leader_election.cc:290] T d6391ca8e3f6476d97c1983eda89cdd2 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:19.040130 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:19.031617 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
W20260807 19:50:19.040983 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 73677.4us
I20260807 19:50:18.920846 13647 consensus_queue.cc:260] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.015244 13202 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:19.045435 8787 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:19.058210 8790 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260807 19:50:19.058918 8790 pending_rounds.cc:85] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting all ops after (but not including) 4
I20260807 19:50:19.059151 8790 pending_rounds.cc:107] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260807 19:50:19.059379 8790 raft_consensus.cc:2889] T 1c01405ca90e4d1ebb91f47316c28aa9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:19.041208 13588 raft_consensus.cc:2749] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260807 19:50:19.060777 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.061266 9020 raft_consensus.cc:1240] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
I20260807 19:50:19.062668 8775 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:50:19.019398 13519 raft_consensus.cc:740] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
W20260807 19:50:19.065132 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 112228us
I20260807 19:50:19.068322 8901 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:19.069263 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 114410us
I20260807 19:50:19.017068 13645 raft_consensus.cc:515] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.075702 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3def68aebfd04eeab681e41691bae18c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:19.083630 9008 raft_consensus.cc:1240] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260807 19:50:19.089298 8856 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.096486 9017 raft_consensus.cc:2393] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 671d8754170646eaac5748745eba58b4 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:18.993665 12927 consensus_queue.cc:237] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.076978 13819 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:19.095361 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d07404635f47998c0f61e0e7f5b732" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.051460 13516 raft_consensus.cc:515] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.105964 8856 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.116586 8775 raft_consensus.cc:1240] T 65e8bf81123c4245acf858d947af51c1 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:19.095814 13695 consensus_queue.cc:1059] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260807 19:50:19.125108 13695 raft_consensus.cc:3055] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Stepping down as leader of term 6
I20260807 19:50:19.125576 13695 raft_consensus.cc:740] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:19.128191 8790 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:19.130266 9027 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10. Ops: []
I20260807 19:50:19.120554 12943 raft_consensus.cc:3055] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:19.140815 12943 raft_consensus.cc:740] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:19.146979 13647 raft_consensus.cc:3060] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:19.148102 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:19.153890 8789 consensus_meta.cc:287] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 0.637s user 0.017s sys 0.005s
I20260807 19:50:19.154260 8789 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:19.094368 13692 raft_consensus.cc:515] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.116645 13824 consensus_queue.cc:1059] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.151s
I20260807 19:50:19.155606 9018 raft_consensus.cc:2468] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 10.
I20260807 19:50:19.158926 13474 consensus_queue.cc:1059] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:19.159307 9017 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260807 19:50:19.160036 13549 raft_consensus.cc:3055] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:19.160545 13549 raft_consensus.cc:740] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:19.156993 8968 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.103976 13666 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:50:19.127360 13394 leader_election.cc:290] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:19.126524 13695 consensus_queue.cc:260] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.152482 13059 raft_consensus.cc:2804] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:19.168182 8903 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:19.147845 13823 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:19.150635 13642 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:19.161523 13549 consensus_queue.cc:260] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.171622 8968 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.203253 8856 leader_election.cc:304] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.207485 13642 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.209427 13727 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:19.210024 13059 raft_consensus.cc:697] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:19.210632 13549 raft_consensus.cc:3060] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:19.211117 13059 consensus_queue.cc:237] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.175294 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.169741 13588 raft_consensus.cc:3055] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:19.219748 8902 raft_consensus.cc:2468] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:19.168443 13666 raft_consensus.cc:493] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8e9ad2b5e2a94d4faff90af735c6d792)
I20260807 19:50:19.173544 13660 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260807 19:50:19.222191 13516 leader_election.cc:290] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:19.223021 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ff5241853a4f548a83179766541c48" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.228994 13660 raft_consensus.cc:493] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.209903 13727 raft_consensus.cc:493] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.209874 13642 leader_election.cc:290] T d8160be756c34002a2ba00cf8deaa10a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:19.204561 13823 raft_consensus.cc:493] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.227504 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b89518940e0d4f7c848bc64e87de7e25" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:19.233143 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.231758 8968 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:19.240379 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.241761 8902 raft_consensus.cc:2410] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260807 19:50:19.243633 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.139644 13519 consensus_queue.cc:260] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.224592 13588 raft_consensus.cc:740] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:19.253144 8903 raft_consensus.cc:3060] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:19.228798 13805 raft_consensus.cc:2804] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:19.226194 13666 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:19.237353 13695 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:19.264233 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.238991 13727 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:19.232707 13645 leader_election.cc:290] T 90fa71b3b19f407d8ede2571544c265c P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:19.236637 13660 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Advancing to term 11
W20260807 19:50:19.265730 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.263087 8856 leader_election.cc:304] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:19.243273 13823 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:19.269304 8968 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.271395 8856 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.278175 13449 raft_consensus.cc:2749] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260807 19:50:19.281764 13863 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.217s user 0.001s sys 0.000s
I20260807 19:50:19.282250 13702 consensus_queue.cc:1059] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260807 19:50:19.282505 13717 consensus_queue.cc:1048] T 92d07404635f47998c0f61e0e7f5b732 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:50:19.283417 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:19.284091 13865 log.cc:927] Time spent T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.062s user 0.000s sys 0.001s
I20260807 19:50:19.284809 8629 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59414 (request call id 74) took 8754 ms. Trace:
I20260807 19:50:19.287076 13802 raft_consensus.cc:3055] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Stepping down as leader of term 8
W20260807 19:50:19.307466 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.288969 13692 leader_election.cc:290] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:19.288798 13645 consensus_queue.cc:1048] T 1ffb51072a064989a2056cfa1484f4a5 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260807 19:50:19.290047 13862 log.cc:927] Time spent T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260807 19:50:19.291052 13680 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:19.291749 13543 raft_consensus.cc:3055] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:19.249822 13819 consensus_queue.cc:1048] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260807 19:50:19.281951 13582 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260807 19:50:19.261983 13805 raft_consensus.cc:493] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.299750 13598 leader_election.cc:290] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:19.304744 8906 raft_consensus.cc:2410] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260807 19:50:19.308634 8902 raft_consensus.cc:2468] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:19.312976 9018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57870 (request call id 1770) took 1144 ms. Trace:
W20260807 19:50:19.313252 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.278677 13394 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260807 19:50:19.277199 13474 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260807 19:50:19.328712 13582 raft_consensus.cc:493] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.332618 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.316570 13680 raft_consensus.cc:493] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.310393 13802 raft_consensus.cc:740] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:19.235976 12943 consensus_queue.cc:260] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.332562 9018 rpcz_store.cc:276] 0807 19:50:18.163695 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:18.611671 (+447976us) service_pool.cc:224] Handling call
0807 19:50:19.307548 (+695877us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b540025c6d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6086da 00007f68805fb927 00007f6879b44834 00007f6879b447c2 00007f6879b446f1 00007f6879b4466c 00007f6879b43922 00007f687993e07c 00007f687993d4b6 00007f687993ce5f
0807 19:50:19.307682 (+ 134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988319616}
I20260807 19:50:19.327126 13519 raft_consensus.cc:3060] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Advancing to term 12
I20260807 19:50:19.331033 13805 raft_consensus.cc:3060] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:19.354620 9890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:19.319056 13543 raft_consensus.cc:740] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:19.356724 8903 raft_consensus.cc:2468] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:19.336190 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:19.335062 13394 raft_consensus.cc:493] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:19.338833 13582 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Advancing to term 5
W20260807 19:50:19.400484 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 126092us
I20260807 19:50:19.401371 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "300f918f185249e6895ab57edd060d31" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.402930 8903 raft_consensus.cc:2468] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
W20260807 19:50:19.366720 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.407140 9018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.350467 13680 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:19.401892 13394 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:19.429739 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "537cbacddca2410db509aff60875a134" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.444139 8903 raft_consensus.cc:1240] T b89518940e0d4f7c848bc64e87de7e25 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260807 19:50:19.358270 13682 consensus_queue.cc:1059] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260807 19:50:19.386961 13202 raft_consensus.cc:515] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.334290 13588 consensus_queue.cc:260] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.305466 8629 rpcz_store.cc:276] 0807 19:50:10.530143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:10.530337 (+ 194us) service_pool.cc:224] Handling call
0807 19:50:19.284526 (+8754189us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0807 19:50:16.007585 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0807 19:50:16.241097 (+233512us) write_op.cc:432] Acquiring schema lock in shared mode
0807 19:50:16.243170 (+ 2073us) write_op.cc:435] Acquired schema lock
0807 19:50:16.246942 (+ 3772us) tablet.cc:662] Decoding operations
0807 19:50:16.568296 (+321354us) write_op.cc:620] Acquiring the partition lock for write op
0807 19:50:16.580642 (+ 12346us) write_op.cc:641] Partition lock acquired for write op
0807 19:50:16.633011 (+ 52369us) tablet.cc:685] Acquiring locks for 4 operations
0807 19:50:16.949313 (+316302us) tablet.cc:701] Row locks acquired
0807 19:50:16.949348 (+ 35us) write_op.cc:260] PREPARE: finished
0807 19:50:16.949512 (+ 164us) write_op.cc:270] Start()
0807 19:50:16.949634 (+ 122us) write_op.cc:276] Timestamp: P: 1786132216949460 usec, L: 0
0807 19:50:16.949655 (+ 21us) op_driver.cc:348] REPLICATION: starting
0807 19:50:16.950346 (+ 691us) log.cc:844] Serialized 1674 byte log entry
0807 19:50:18.040752 (+1090406us) op_driver.cc:464] REPLICATION: finished
0807 19:50:18.208546 (+167794us) write_op.cc:301] APPLY: starting
0807 19:50:18.208715 (+ 169us) tablet.cc:1367] starting BulkCheckPresence
0807 19:50:18.208973 (+ 258us) tablet.cc:1370] finished BulkCheckPresence
0807 19:50:18.209127 (+ 154us) tablet.cc:1372] starting ApplyRowOperation cycle
0807 19:50:18.213013 (+ 3886us) tablet.cc:1383] finished ApplyRowOperation cycle
0807 19:50:18.213364 (+ 351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0807 19:50:18.213468 (+ 104us) write_op.cc:312] APPLY: finished
0807 19:50:18.264954 (+ 51486us) log.cc:844] Serialized 41 byte log entry
0807 19:50:18.285028 (+ 20074us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:18.285244 (+ 216us) write_op.cc:454] Released schema lock
0807 19:50:18.285302 (+ 58us) write_op.cc:341] FINISH: Updating metrics
0807 19:50:18.285964 (+ 662us) write_op.cc:489] Releasing partition, row and schema locks
0807 19:50:18.285997 (+ 33us) write_op.cc:454] Released schema lock
0807 19:50:19.167268 (+881271us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540011a8d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b60170a 00007f687b5872f4 00007f687b5872a3 00007f687b587218 00007f687b5871d3 00007f687b5864bc 00007f687b5b01f6 00007f687b59809b 00007f687b58277d 00007f687b586155
Metrics: {"mutex_wait_us":1396,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":126575,"apply.run_cpu_time_us":4374,"apply.run_wall_time_us":82376,"num_ops":4,"prepare.queue_time_us":470187,"prepare.run_cpu_time_us":41656,"prepare.run_wall_time_us":1243606,"raft.queue_time_us":203608,"raft.run_cpu_time_us":21255,"raft.run_wall_time_us":1293352,"replication_time_us":1066194,"spinlock_wait_cycles":250087808,"thread_start_us":729544,"threads_started":4,"wal-append.queue_time_us":320821,"wal-append.run_cpu_time_us":46071,"wal-append.run_wall_time_us":2024493}]]}
I20260807 19:50:19.473658 13802 consensus_queue.cc:260] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.475030 13802 raft_consensus.cc:3060] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:50:19.475454 13867 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20260807 19:50:19.477650 13543 consensus_queue.cc:260] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.478945 13543 raft_consensus.cc:3060] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:19.481910 8972 leader_election.cc:304] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:19.482831 13516 raft_consensus.cc:2749] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:19.486618 8908 raft_consensus.cc:1240] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260807 19:50:19.489269 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.490635 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:19.492805 13805 raft_consensus.cc:515] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.495231 13805 leader_election.cc:290] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:19.501842 13202 leader_election.cc:290] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:19.502614 9027 raft_consensus.cc:1240] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 6. Current term is 7. Ops: []
W20260807 19:50:19.505036 13878 log.cc:927] Time spent T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.052s user 0.001s sys 0.000s
I20260807 19:50:19.507786 8911 raft_consensus.cc:1275] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260807 19:50:19.509709 13202 raft_consensus.cc:493] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:19.510454 13202 raft_consensus.cc:515] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.513517 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.514283 8911 raft_consensus.cc:2468] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
W20260807 19:50:19.515527 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.518873 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "915c924a58ad4f3cbf22affe06485a15" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.519165 9018 raft_consensus.cc:1275] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260807 19:50:19.519101 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65e8bf81123c4245acf858d947af51c1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:19.519910 9021 raft_consensus.cc:1275] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Refusing update from remote peer 8e9ad2b5e2a94d4faff90af735c6d792: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260807 19:50:19.530812 13719 consensus_meta.cc:287] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.534s user 0.010s sys 0.002s
I20260807 19:50:19.499658 13660 raft_consensus.cc:515] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:19.555522 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 150301us
I20260807 19:50:19.566768 9008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a898110875ba40ab8e512e124aff4fb2" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.574249 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c01405ca90e4d1ebb91f47316c28aa9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.583478 9017 raft_consensus.cc:2393] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 11: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:19.592896 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.595103 8913 raft_consensus.cc:1240] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Rejecting Update request from peer 8e9ad2b5e2a94d4faff90af735c6d792 for earlier term 8. Current term is 9. Ops: []
I20260807 19:50:19.588099 8968 leader_election.cc:304] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.569561 12927 consensus_queue.cc:1048] T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260807 19:50:19.611589 9008 raft_consensus.cc:2468] T a898110875ba40ab8e512e124aff4fb2 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
W20260807 19:50:19.632954 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 67841.1us
I20260807 19:50:19.647214 8901 raft_consensus.cc:2468] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:19.648044 13603 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:19.649994 13876 log.cc:927] Time spent T fe08dfdbfbdb4fd08342efd539590bf2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.184s user 0.000s sys 0.001s
W20260807 19:50:19.650748 13872 log.cc:927] Time spent T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5: Append to log took a long time: real 0.135s user 0.000s sys 0.000s
I20260807 19:50:19.651387 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:19.652554 13394 raft_consensus.cc:515] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.655480 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:19.626014 13202 leader_election.cc:290] T d54b306954094de7a87c17dbbf6ef21f P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:19.722990 8789 raft_consensus.cc:3060] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:19.657649 8787 raft_consensus.cc:1240] T fced846726734919ac0881f9a26379ba P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
W20260807 19:50:19.658839 13880 log.cc:927] Time spent T a898110875ba40ab8e512e124aff4fb2 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260807 19:50:19.734050 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
W20260807 19:50:19.652581 13889 log.cc:927] Time spent T 7572f4807051425c936e926fa8c8d74e P 671d8754170646eaac5748745eba58b4: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
I20260807 19:50:19.661465 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d0dfd76f4dc484499bc12ddf5871c48" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.661875 8783 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260807 19:50:19.634238 13727 raft_consensus.cc:515] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.665162 13666 raft_consensus.cc:515] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.666009 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:19.669577 8775 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260807 19:50:19.648808 13474 raft_consensus.cc:2804] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:19.672537 8790 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260807 19:50:19.672731 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.648767 13603 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:19.682174 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.664168 9022 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260807 19:50:19.728614 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:19.728947 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.741431 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:19.757221 8779 raft_consensus.cc:3060] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:19.746975 8910 raft_consensus.cc:1275] T f2b65bfd5b3948cea9db5090e802132d P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260807 19:50:19.757143 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.747884 8915 raft_consensus.cc:2393] T 9d0dfd76f4dc484499bc12ddf5871c48 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 9: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:19.757061 9020 raft_consensus.cc:1240] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260807 19:50:19.761075 13603 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:19.762288 13666 leader_election.cc:290] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:19.764348 13727 leader_election.cc:290] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:19.769019 13582 raft_consensus.cc:515] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.771441 13582 leader_election.cc:290] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:19.775744 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:19.778054 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:19.778479 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:19.778957 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.769680 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:19.787658 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:19.790712 13474 raft_consensus.cc:697] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:19.792173 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:19.802162 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:19.840304 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.652704 13881 consensus_queue.cc:1048] T 2e99139b6d2d4da082b4017e95030a69 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.179s
W20260807 19:50:19.825568 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 51744.2us
W20260807 19:50:19.779561 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.791812 9019 raft_consensus.cc:2468] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 9.
I20260807 19:50:19.828295 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.842730 8789 raft_consensus.cc:2468] T 8977afab19724738a872c7880e363557 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
I20260807 19:50:19.844686 9021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7abc25df9cc247e8b0354c9758c11208" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:19.845526 9021 raft_consensus.cc:3060] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Advancing to term 5
I20260807 19:50:19.844123 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.846760 8902 raft_consensus.cc:2410] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260807 19:50:19.851585 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.852113 8788 raft_consensus.cc:1240] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 10. Current term is 11. Ops: []
I20260807 19:50:19.849610 8915 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:19.856154 8910 raft_consensus.cc:2393] T afb6953f02d1486aaee40c8901954290 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8e9ad2b5e2a94d4faff90af735c6d792 in current term 9: Already voted for candidate a38f4d6088cf4c8cb510200ca0555801 in this term.
I20260807 19:50:19.855234 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.866835 8856 leader_election.cc:304] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:19.855261 9022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95f757dd3f63420fad091c3802390158" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:19.868623 9022 raft_consensus.cc:3060] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Advancing to term 11
W20260807 19:50:19.858230 13788 consensus_meta.cc:287] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792: Time spent flushing consensus metadata: real 1.269s user 0.017s sys 0.008s
W20260807 19:50:19.853995 13823 consensus_meta.cc:287] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.551s user 0.011s sys 0.004s
I20260807 19:50:19.850057 9885 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:19.868328 13516 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:19.894629 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:19.895292 8779 raft_consensus.cc:2468] T 06dc193f54374639a09c9b5827ff15f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:19.881616 13666 raft_consensus.cc:2804] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:19.868103 13819 consensus_queue.cc:1059] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260807 19:50:19.881445 13394 leader_election.cc:290] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:19.875409 13615 consensus_queue.cc:1059] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260807 19:50:19.909127 9023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e99139b6d2d4da082b4017e95030a69" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.889437 13549 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:19.879989 13788 raft_consensus.cc:515] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.888789 13519 consensus_queue.cc:1048] T 7f72bb6e69514b33881da72833a29378 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260807 19:50:19.882740 13823 raft_consensus.cc:515] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.903265 13543 raft_consensus.cc:493] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:19.900583 13202 consensus_queue.cc:1059] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260807 19:50:19.917881 13666 raft_consensus.cc:697] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [term 10 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:19.927913 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:19.932420 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.924700 13549 raft_consensus.cc:515] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.933403 9015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb6953f02d1486aaee40c8901954290" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:19.933295 13474 consensus_queue.cc:237] T afb6953f02d1486aaee40c8901954290 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:19.919014 13660 leader_election.cc:290] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:19.920578 13890 consensus_queue.cc:1048] T f2b65bfd5b3948cea9db5090e802132d P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260807 19:50:19.890321 13898 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260807 19:50:19.937225 8788 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:19.938292 8789 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 19:50:19.925607 8915 raft_consensus.cc:1275] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260807 19:50:19.944209 8902 raft_consensus.cc:2393] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a38f4d6088cf4c8cb510200ca0555801 in current term 10: Already voted for candidate 671d8754170646eaac5748745eba58b4 in this term.
I20260807 19:50:19.915132 13695 consensus_queue.cc:1048] T 82ff5241853a4f548a83179766541c48 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260807 19:50:19.899001 13516 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:19.936532 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbfe42e0961647279ca7973da14c62a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:19.946719 8778 raft_consensus.cc:1240] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Rejecting Update request from peer a38f4d6088cf4c8cb510200ca0555801 for earlier term 5. Current term is 6. Ops: []
W20260807 19:50:20.215536 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:19.944746 13102 raft_consensus.cc:3055] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 LEADER]: Stepping down as leader of term 10
I20260807 19:50:20.451346 13102 raft_consensus.cc:740] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.103538 13788 leader_election.cc:290] T ed0dd3f602c145219dc8717e182ff568 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 6 election: Requested vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:20.315794 9027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57870 (request call id 1794) took 1017 ms. Trace:
I20260807 19:50:20.454685 9027 rpcz_store.cc:276] 0807 19:50:19.298143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0807 19:50:19.306559 (+ 8416us) service_pool.cc:224] Handling call
0807 19:50:19.327926 (+ 21367us) raft_consensus.cc:1471] Updating replica for 0 ops
0807 19:50:19.503029 (+175103us) raft_consensus.cc:1697] Filling consensus response to leader.
0807 19:50:20.315535 (+812506us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54000695d4. stack: 00007f68784cbcf3 00007f68784cbbf0 00007f6877e56574 00007f6881bec6c4 00007f6881bec659 00007f687b56d951 00007f687b6051e1 00007f687b603c1e 00007f68805fa229 00007f6879b3fd34 00007f6879b3fcc2 00007f6879b3fbf1 00007f6879b3fb6c 00007f6879b3ee22 00007f687993e07c 00007f687993d4b6
0807 19:50:20.315723 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401324416}
W20260807 19:50:20.457500 8968 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.015321 13823 leader_election.cc:290] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:20.450385 13516 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:19.961802 13680 consensus_meta.cc:287] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801: Time spent flushing consensus metadata: real 0.525s user 0.011s sys 0.002s
I20260807 19:50:19.953791 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:20.461016 13680 raft_consensus.cc:515] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.461562 8776 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:20.452062 13588 raft_consensus.cc:3060] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:19.946578 13543 raft_consensus.cc:515] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:20.109144 6315 auto_leader_rebalancer.cc:153] leader rebalance for table test-workload
I20260807 19:50:20.466917 13543 leader_election.cc:290] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:20.011632 13824 raft_consensus.cc:3055] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:20.467849 13824 raft_consensus.cc:740] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 39975243456d4452b4a4870ac7ebafc5, State: Running, Role: LEADER
I20260807 19:50:20.134315 9022 raft_consensus.cc:2468] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 11.
I20260807 19:50:20.469434 8788 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:20.448467 12943 raft_consensus.cc:3060] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:20.376361 13598 raft_consensus.cc:3055] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Stepping down as leader of term 9
I20260807 19:50:20.472587 13598 raft_consensus.cc:740] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:20.473783 13598 consensus_queue.cc:260] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:20.475414 13598 raft_consensus.cc:3060] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:50:20.475899 8740 leader_election.cc:336] T be89699a1aff43f1a97b936b3154c2d5 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801): Timed out: RequestConsensusVote RPC to 127.6.42.195:35801 timed out after 2.217s (SENT)
I20260807 19:50:19.953436 13202 raft_consensus.cc:3055] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Stepping down as leader of term 5
I20260807 19:50:20.468986 13824 consensus_queue.cc:260] T 413683c7378447239874762d812d75e5 P 39975243456d4452b4a4870ac7ebafc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:20.470955 8908 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.011s sys 0.007s
W20260807 19:50:20.107817 13888 log.cc:927] Time spent T 537cbacddca2410db509aff60875a134 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.355s user 0.001s sys 0.000s
W20260807 19:50:19.956049 9838 leader_election.cc:336] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 2.127s (SENT)
I20260807 19:50:20.499675 9838 leader_election.cc:304] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:20.499116 8908 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s user 0.011s sys 0.007s
I20260807 19:50:20.477499 8779 raft_consensus.cc:3060] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:20.478718 8910 raft_consensus.cc:1275] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260807 19:50:20.500696 8910 pending_rounds.cc:85] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 4
I20260807 19:50:20.500985 8910 pending_rounds.cc:107] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260807 19:50:20.501219 8910 raft_consensus.cc:2889] T 2e99139b6d2d4da082b4017e95030a69 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:20.119539 13666 consensus_queue.cc:237] T d0e0dcc92bf04b66a4b58d4e68b01703 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.502671 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8977afab19724738a872c7880e363557" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:20.503983 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06dc193f54374639a09c9b5827ff15f4" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:20.507434 8787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:20.508210 8787 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:20.509214 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90fa71b3b19f407d8ede2571544c265c" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:20.510164 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.507432 8910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e0472ef12247a4a432a786e2656de3" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:20.513386 8910 raft_consensus.cc:3060] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:20.479079 8903 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "133c01ab82cf4ef3b89a9649b0abf969" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:20.452735 13102 consensus_queue.cc:260] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:20.265127 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.186905 9021 raft_consensus.cc:2468] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 5.
W20260807 19:50:20.251621 13905 log.cc:927] Time spent T 1ffb51072a064989a2056cfa1484f4a5 P 8e9ad2b5e2a94d4faff90af735c6d792: Append to log took a long time: real 0.150s user 0.000s sys 0.001s
I20260807 19:50:20.521646 8778 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260807 19:50:20.522281 13603 raft_consensus.cc:476] T 95f757dd3f63420fad091c3802390158 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 LEADER]: Not starting pre-election -- already a leader
I20260807 19:50:20.522888 9020 raft_consensus.cc:1240] T 133c01ab82cf4ef3b89a9649b0abf969 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 9. Current term is 10. Ops: []
W20260807 19:50:20.481191 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.523517 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "301b269c068f46ebae7e1c1d55e501f1" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801"
I20260807 19:50:20.481137 8911 raft_consensus.cc:1275] T 300f918f185249e6895ab57edd060d31 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Refusing update from remote peer 39975243456d4452b4a4870ac7ebafc5: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260807 19:50:20.481873 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.405616 8632 catalog_manager.cc:5697] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:50:20.450320 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.452853 8915 pending_rounds.cc:85] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 3
I20260807 19:50:20.485589 8968 leader_election.cc:304] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:20.487809 8776 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
W20260807 19:50:20.500154 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.524138 9019 raft_consensus.cc:3060] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
I20260807 19:50:20.525552 8915 pending_rounds.cc:107] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260807 19:50:20.510402 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "413683c7378447239874762d812d75e5" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:20.526252 8915 raft_consensus.cc:2889] T 1ffb51072a064989a2056cfa1484f4a5 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260807 19:50:20.525553 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.527387 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.527742 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.528059 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.528378 8856 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.534595 13102 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:20.534860 13660 raft_consensus.cc:2804] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader election won for term 11
I20260807 19:50:20.535446 13660 raft_consensus.cc:697] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.536029 8911 raft_consensus.cc:1275] T 1c01405ca90e4d1ebb91f47316c28aa9 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260807 19:50:20.536612 13660 consensus_queue.cc:237] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:20.545696 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 69862.2us
I20260807 19:50:20.536615 8632 catalog_manager.cc:5697] T fe08dfdbfbdb4fd08342efd539590bf2 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 6 to 10, leader changed from 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 10 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.538951 8856 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:20.534497 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 49056.7us
I20260807 19:50:20.539698 9886 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:20.541422 13823 raft_consensus.cc:493] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:20.541695 13615 raft_consensus.cc:2749] T a898110875ba40ab8e512e124aff4fb2 P 39975243456d4452b4a4870ac7ebafc5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260807 19:50:20.541905 13582 consensus_queue.cc:1059] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 19:50:20.489279 13202 raft_consensus.cc:740] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.542593 13819 consensus_queue.cc:1059] T 06dc193f54374639a09c9b5827ff15f4 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260807 19:50:20.543622 9008 raft_consensus.cc:1240] T 413683c7378447239874762d812d75e5 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Rejecting Update request from peer 39975243456d4452b4a4870ac7ebafc5 for earlier term 5. Current term is 6. Ops: []
W20260807 19:50:20.540333 9838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 584302us
I20260807 19:50:20.537520 8906 raft_consensus.cc:1275] T 7f72bb6e69514b33881da72833a29378 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260807 19:50:20.539691 8915 raft_consensus.cc:1275] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260807 19:50:20.555469 8968 leader_election.cc:304] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:20.550549 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d54b306954094de7a87c17dbbf6ef21f" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
W20260807 19:50:20.546315 8740 connection.cc:256] RPC call timeout handler was delayed by -2.0034172600135207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260807 19:50:20.576993 8740 connection.cc:256] RPC call timeout handler was delayed by -1.9395172600634396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260807 19:50:20.577494 8740 connection.cc:256] RPC call timeout handler was delayed by -1.9026172601152211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1626s for a response
W20260807 19:50:20.577924 8740 connection.cc:256] RPC call timeout handler was delayed by -1.8684172602370381s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260807 19:50:20.578361 8740 connection.cc:256] RPC call timeout handler was delayed by -1.5318172601982951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
I20260807 19:50:20.576330 8915 pending_rounds.cc:85] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Aborting all ops after (but not including) 5
W20260807 19:50:20.578701 8740 connection.cc:256] RPC call timeout handler was delayed by -1.4373172600753605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260807 19:50:20.579100 8915 pending_rounds.cc:107] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4: Aborting uncommitted NO_OP operation due to leader change: 8.6
W20260807 19:50:20.579231 8740 connection.cc:256] RPC call timeout handler was delayed by -1.3851172600407153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22010000000000002s for a response
I20260807 19:50:20.579557 8915 raft_consensus.cc:2889] T 82ff5241853a4f548a83179766541c48 P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260807 19:50:20.581701 8908 raft_consensus.cc:2468] T d54b306954094de7a87c17dbbf6ef21f P 671d8754170646eaac5748745eba58b4 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:20.569293 8788 raft_consensus.cc:2468] T 0ba47e79caa54c409c53304a66ce0160 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:20.572985 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ba47e79caa54c409c53304a66ce0160" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:20.581921 8911 raft_consensus.cc:1275] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260807 19:50:20.586663 9019 raft_consensus.cc:2468] T 301b269c068f46ebae7e1c1d55e501f1 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 10.
I20260807 19:50:20.588681 8740 leader_election.cc:304] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:20.588812 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:20.597914 13603 raft_consensus.cc:2749] T f02f58b5ca51497592868892f144492a P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:20.542488 13900 consensus_queue.cc:1048] T 300f918f185249e6895ab57edd060d31 P 39975243456d4452b4a4870ac7ebafc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260807 19:50:20.599233 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:20.589965 8632 catalog_manager.cc:5697] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 2 to 7, leader changed from a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195) to 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193). New cstate: current_term: 7 leader_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } }
I20260807 19:50:20.608074 8779 raft_consensus.cc:2468] T fbfe42e0961647279ca7973da14c62a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:20.575327 13823 raft_consensus.cc:515] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.587390 13728 raft_consensus.cc:2804] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:20.575664 9886 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 7.
I20260807 19:50:20.620767 8904 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:20.621187 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "479d60d809ec458397b1360e820ed150" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:20.629963 8968 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:20.656716 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.659029 8632 catalog_manager.cc:5697] T be89699a1aff43f1a97b936b3154c2d5 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.660722 8632 catalog_manager.cc:5697] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.662262 8740 leader_election.cc:304] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:20.673491 13890 raft_consensus.cc:2804] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260807 19:50:20.674069 13890 raft_consensus.cc:493] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:20.674422 13890 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:20.618196 12927 consensus_queue.cc:1059] T ed0dd3f602c145219dc8717e182ff568 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260807 19:50:20.676697 8632 catalog_manager.cc:5697] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:50:20.679004 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.682771 8910 raft_consensus.cc:2468] T 26e0472ef12247a4a432a786e2656de3 P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 11.
I20260807 19:50:20.691383 8632 catalog_manager.cc:5697] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 7 to 9, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 9 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.695859 8632 catalog_manager.cc:5697] T 537cbacddca2410db509aff60875a134 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 5 to 6, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 6 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
W20260807 19:50:20.705904 8972 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.708154 9850 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 167228us
W20260807 19:50:20.709929 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 94167.1us
W20260807 19:50:20.710506 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.629269 13059 consensus_queue.cc:1048] T 3def68aebfd04eeab681e41691bae18c P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
W20260807 19:50:20.712598 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.716696 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 86840.4us
I20260807 19:50:20.717933 8632 catalog_manager.cc:5697] T 65e8bf81123c4245acf858d947af51c1 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 7 to 8, leader changed from 671d8754170646eaac5748745eba58b4 (127.6.42.194) to a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195). New cstate: current_term: 8 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.697607 13603 raft_consensus.cc:2749] T 7f72bb6e69514b33881da72833a29378 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:20.725996 9886 raft_consensus.cc:1275] T 2fe2de08a4ba49eea927f371b4bf0e5f P 39975243456d4452b4a4870ac7ebafc5 [term 9 FOLLOWER]: Refusing update from remote peer a38f4d6088cf4c8cb510200ca0555801: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260807 19:50:20.713439 13582 raft_consensus.cc:2804] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader election won for term 11
I20260807 19:50:20.720844 13543 raft_consensus.cc:493] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:20.742365 8632 catalog_manager.cc:5697] T 4ec2edf1da424177aaad8b93086118b1 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.723639 8968 leader_election.cc:304] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:20.754182 8968 leader_election.cc:304] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:20.754485 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 215624us
I20260807 19:50:20.714107 13666 raft_consensus.cc:3055] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Stepping down as leader of term 8
I20260807 19:50:20.755968 8968 leader_election.cc:304] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:20.756904 8856 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:20.760046 8779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef1e484df4724d9896a8f46b89923261" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:20.761045 8779 raft_consensus.cc:2468] T ef1e484df4724d9896a8f46b89923261 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:20.768920 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49088520f262493898ad206e1e077f2b" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792"
I20260807 19:50:20.769865 8776 raft_consensus.cc:3060] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Advancing to term 9
W20260807 19:50:20.769074 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 60196.9us
I20260807 19:50:20.779982 8790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be133eef91634be592094b95357ee200" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:20.756876 13728 raft_consensus.cc:697] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.740592 13823 leader_election.cc:290] T b8f9a221ee034d3b86101183048438ff P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
W20260807 19:50:20.770493 13885 log.cc:927] Time spent T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.053s user 0.000s sys 0.002s
W20260807 19:50:20.786679 8854 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.723063 12927 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260807 19:50:20.773751 13881 raft_consensus.cc:2804] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader election won for term 11
I20260807 19:50:20.772058 13666 raft_consensus.cc:740] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:20.771812 13680 leader_election.cc:290] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:20.798278 8632 catalog_manager.cc:5697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a38f4d6088cf4c8cb510200ca0555801" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } health_report { overall_health: HEALTHY } } }
I20260807 19:50:20.715282 13202 consensus_queue.cc:260] T fced846726734919ac0881f9a26379ba P a38f4d6088cf4c8cb510200ca0555801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.806627 8776 raft_consensus.cc:2468] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 9.
I20260807 19:50:20.790485 13645 raft_consensus.cc:2804] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:20.770154 13543 raft_consensus.cc:515] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:20.842084 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 50924us
W20260807 19:50:20.810910 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 64446us
I20260807 19:50:20.843463 8972 leader_election.cc:304] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:20.836854 13666 consensus_queue.cc:260] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.809082 13059 raft_consensus.cc:493] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:20.799472 13692 raft_consensus.cc:2804] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader election won for term 7
I20260807 19:50:20.797381 13823 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:20.848564 8968 leader_election.cc:304] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:20.857276 8740 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:20.862800 13924 raft_consensus.cc:2804] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Leader election won for term 5
I20260807 19:50:20.863492 13924 raft_consensus.cc:697] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [term 5 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:20.855216 13059 raft_consensus.cc:515] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.865474 8972 leader_election.cc:304] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:20.849309 13881 raft_consensus.cc:697] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.852658 13935 raft_consensus.cc:493] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:20.907215 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 50025.4us
I20260807 19:50:20.853819 13895 consensus_queue.cc:1048] T 2fe2de08a4ba49eea927f371b4bf0e5f P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260807 19:50:20.897984 13603 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:20.898896 13666 raft_consensus.cc:3060] T 0ba47e79caa54c409c53304a66ce0160 P 671d8754170646eaac5748745eba58b4 [term 8 FOLLOWER]: Advancing to term 9
I20260807 19:50:20.857452 13823 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:20.915414 8849 connection.cc:256] RPC call timeout handler was delayed by -1.4926659592892975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
I20260807 19:50:20.896585 13645 raft_consensus.cc:697] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.903867 13582 raft_consensus.cc:697] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [term 11 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
W20260807 19:50:20.930644 8849 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:20.931213 8849 connection.cc:256] RPC call timeout handler was delayed by -1.3297659591771662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
W20260807 19:50:20.931595 8849 connection.cc:256] RPC call timeout handler was delayed by -1.3252659591380507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.221s for a response
W20260807 19:50:20.931870 8849 connection.cc:256] RPC call timeout handler was delayed by -1.290015771985054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.328s for a response
W20260807 19:50:20.932132 8849 connection.cc:256] RPC call timeout handler was delayed by -1.2649659591261297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2277s for a response
W20260807 19:50:20.932454 8849 connection.cc:256] RPC call timeout handler was delayed by -1.2352179491426796s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260807 19:50:20.932570 13692 raft_consensus.cc:697] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [term 7 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:20.933936 13692 consensus_queue.cc:237] T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
W20260807 19:50:20.932806 8849 connection.cc:256] RPC call timeout handler was delayed by -1.0206866869702935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
W20260807 19:50:20.934729 9848 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.934177 13582 consensus_queue.cc:237] T 95f757dd3f63420fad091c3802390158 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.916988 13936 raft_consensus.cc:2749] T 95f757dd3f63420fad091c3802390158 P a38f4d6088cf4c8cb510200ca0555801 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260807 19:50:20.950393 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 101914us
I20260807 19:50:20.903002 13935 raft_consensus.cc:515] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.958369 13935 leader_election.cc:290] T 7572f4807051425c936e926fa8c8d74e P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:20.960191 13823 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:20.913513 13728 consensus_queue.cc:237] T 06dc193f54374639a09c9b5827ff15f4 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.961848 13881 consensus_queue.cc:237] T 26e0472ef12247a4a432a786e2656de3 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.897321 13924 consensus_queue.cc:237] T 7abc25df9cc247e8b0354c9758c11208 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:20.964236 13543 leader_election.cc:290] T 17510c7d48eb44ceb3d8c645ceaa28fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:20.965399 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 100046us
I20260807 19:50:20.966522 8972 leader_election.cc:304] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:20.930259 13680 raft_consensus.cc:2804] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260807 19:50:20.978451 8740 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:20.932389 13660 raft_consensus.cc:2804] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:20.947388 13549 leader_election.cc:290] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:20.994724 8968 leader_election.cc:304] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:20.973739 13059 leader_election.cc:290] T 8666b7cb7a05408a8223317d32823bc9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:21.002488 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260807 19:50:21.003629 8853 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:20.978044 13680 raft_consensus.cc:493] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:20.983500 13516 raft_consensus.cc:2804] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260807 19:50:21.009716 13615 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:21.010358 8972 leader_election.cc:304] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:20.990638 13948 raft_consensus.cc:2749] T 49088520f262493898ad206e1e077f2b P 8e9ad2b5e2a94d4faff90af735c6d792 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260807 19:50:21.017551 8740 leader_election.cc:304] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792; no voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801
W20260807 19:50:21.013610 8856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 257150us
I20260807 19:50:21.005247 13935 raft_consensus.cc:2749] T be133eef91634be592094b95357ee200 P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:20.996066 8968 leader_election.cc:304] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801; no voters:
W20260807 19:50:20.999362 8971 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:21.008462 13680 raft_consensus.cc:3060] T ef1e484df4724d9896a8f46b89923261 P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Advancing to term 10
W20260807 19:50:20.991430 8849 leader_election.cc:336] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.624s (SENT)
I20260807 19:50:21.012233 9885 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:21.012284 13516 raft_consensus.cc:493] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 671d8754170646eaac5748745eba58b4)
I20260807 19:50:21.015591 13946 raft_consensus.cc:493] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:21.029872 8856 leader_election.cc:304] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:21.017045 13660 raft_consensus.cc:697] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [term 10 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:21.024626 8849 leader_election.cc:304] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:21.015616 13615 raft_consensus.cc:515] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:21.023777 13964 raft_consensus.cc:493] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:21.060530 13059 consensus_queue.cc:1048] T d6391ca8e3f6476d97c1983eda89cdd2 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Connected to new peer: Peer: permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260807 19:50:21.063472 8972 leader_election.cc:304] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5, 671d8754170646eaac5748745eba58b4
I20260807 19:50:21.064869 8972 leader_election.cc:304] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:21.065892 13692 raft_consensus.cc:2749] T 92d07404635f47998c0f61e0e7f5b732 P a38f4d6088cf4c8cb510200ca0555801 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260807 19:50:21.066744 13692 raft_consensus.cc:2804] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260807 19:50:21.067178 13692 raft_consensus.cc:493] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:50:21.067504 13692 raft_consensus.cc:3060] T f4ace421d3564015bfd7eb6cff4b78fc P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Advancing to term 7
I20260807 19:50:21.058045 13102 raft_consensus.cc:2804] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 9 FOLLOWER]: Leader election won for term 9
I20260807 19:50:21.070570 8904 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4ace421d3564015bfd7eb6cff4b78fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:21.070786 13102 raft_consensus.cc:697] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [term 9 LEADER]: Becoming Leader. State: Replica: a38f4d6088cf4c8cb510200ca0555801, State: Running, Role: LEADER
I20260807 19:50:21.071321 8904 raft_consensus.cc:2468] T f4ace421d3564015bfd7eb6cff4b78fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:21.072789 8911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:21.073169 8915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17510c7d48eb44ceb3d8c645ceaa28fc" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:21.072815 8913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "138f0e8a31994c549dd743c9ba49d6ee" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:21.074025 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ec2edf1da424177aaad8b93086118b1" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:21.032673 13516 raft_consensus.cc:3060] T fb33dc9f5a49409bb8817073ec1143da P a38f4d6088cf4c8cb510200ca0555801 [term 3 FOLLOWER]: Advancing to term 4
I20260807 19:50:21.074774 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:21.075261 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10901e14a5e8471291ad1d7c203b6740" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:21.047655 13948 raft_consensus.cc:493] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:21.008572 13645 consensus_queue.cc:237] T 0ba47e79caa54c409c53304a66ce0160 P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.076177 13948 raft_consensus.cc:515] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:21.039362 13946 raft_consensus.cc:515] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
W20260807 19:50:21.057869 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 61864.3us
W20260807 19:50:21.075528 8970 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:21.080240 8915 raft_consensus.cc:3060] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Advancing to term 7
W20260807 19:50:21.064802 13943 log.cc:927] Time spent T 1c01405ca90e4d1ebb91f47316c28aa9 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260807 19:50:21.082834 8906 raft_consensus.cc:2468] T 10901e14a5e8471291ad1d7c203b6740 P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 4.
I20260807 19:50:21.086736 8972 leader_election.cc:304] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a38f4d6088cf4c8cb510200ca0555801; no voters: 671d8754170646eaac5748745eba58b4, 8e9ad2b5e2a94d4faff90af735c6d792
W20260807 19:50:21.088660 8740 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.195:35801, user_credentials={real_user=slave}} blocked reactor thread for 73859.8us
I20260807 19:50:21.088965 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0e0dcc92bf04b66a4b58d4e68b01703" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
W20260807 19:50:21.061071 13598 consensus_meta.cc:287] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4: Time spent flushing consensus metadata: real 0.585s user 0.012s sys 0.000s
I20260807 19:50:21.078508 13948 leader_election.cc:290] T 7e8d547d81964989bb6cd1b677c971c4 P 8e9ad2b5e2a94d4faff90af735c6d792 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
I20260807 19:50:21.090112 9017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:21.039870 13788 raft_consensus.cc:2749] T e10d78176f8545489a4700b7bd08fec3 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:21.043376 13819 raft_consensus.cc:2804] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election won for term 10
I20260807 19:50:21.073971 13615 leader_election.cc:290] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659), a38f4d6088cf4c8cb510200ca0555801 (127.6.42.195:35801)
I20260807 19:50:21.082672 9885 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 39975243456d4452b4a4870ac7ebafc5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 4.
I20260807 19:50:21.071805 13102 consensus_queue.cc:237] T 49088520f262493898ad206e1e077f2b P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.081063 13895 raft_consensus.cc:2749] T 2d69c395b87b43c0bf7c3d1efb63e699 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260807 19:50:21.102576 8902 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "39975243456d4452b4a4870ac7ebafc5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4" is_pre_election: true
I20260807 19:50:21.132575 8902 raft_consensus.cc:2468] T fb33dc9f5a49409bb8817073ec1143da P 671d8754170646eaac5748745eba58b4 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39975243456d4452b4a4870ac7ebafc5 in term 4.
W20260807 19:50:21.132990 8736 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:21.107667 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e8d547d81964989bb6cd1b677c971c4" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5" is_pre_election: true
I20260807 19:50:21.126514 9019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8f9a221ee034d3b86101183048438ff" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:21.127597 9022 raft_consensus.cc:1240] T 8977afab19724738a872c7880e363557 P a38f4d6088cf4c8cb510200ca0555801 [term 11 LEADER]: Rejecting Update request from peer 671d8754170646eaac5748745eba58b4 for earlier term 10. Current term is 11. Ops: []
I20260807 19:50:21.128206 9020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb33dc9f5a49409bb8817073ec1143da" candidate_uuid: "671d8754170646eaac5748745eba58b4" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a38f4d6088cf4c8cb510200ca0555801" is_pre_election: true
I20260807 19:50:21.129406 9008 raft_consensus.cc:1275] T d0e0dcc92bf04b66a4b58d4e68b01703 P a38f4d6088cf4c8cb510200ca0555801 [term 10 FOLLOWER]: Refusing update from remote peer 671d8754170646eaac5748745eba58b4: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260807 19:50:21.108182 8908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be89699a1aff43f1a97b936b3154c2d5" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:21.135741 8908 raft_consensus.cc:3060] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Advancing to term 11
I20260807 19:50:21.137931 13946 leader_election.cc:290] T bff76e2c6d6f4e35bf33640e6f4312a0 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
I20260807 19:50:21.055157 13964 raft_consensus.cc:515] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.107246 8906 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fced846726734919ac0881f9a26379ba" candidate_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "671d8754170646eaac5748745eba58b4"
I20260807 19:50:21.144501 8906 raft_consensus.cc:3060] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Advancing to term 6
I20260807 19:50:21.143862 8903 raft_consensus.cc:2468] T 133c01ab82cf4ef3b89a9649b0abf969 P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
W20260807 19:50:21.145725 8849 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 159615us
W20260807 19:50:21.146833 8849 leader_election.cc:336] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185): Timed out: RequestConsensusVote RPC to 127.6.42.193:33185 timed out after 1.696s (SENT)
I20260807 19:50:21.047341 13890 raft_consensus.cc:515] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:21.147225 8849 leader_election.cc:304] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 671d8754170646eaac5748745eba58b4; no voters: 8e9ad2b5e2a94d4faff90af735c6d792, a38f4d6088cf4c8cb510200ca0555801
I20260807 19:50:21.148176 8908 raft_consensus.cc:2468] T be89699a1aff43f1a97b936b3154c2d5 P 671d8754170646eaac5748745eba58b4 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 11.
I20260807 19:50:21.143831 13964 leader_election.cc:290] T 80eb998e349c4e2885504025672242a7 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 671d8754170646eaac5748745eba58b4 (127.6.42.194:37659)
W20260807 19:50:21.149055 8968 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.42.193:33185, user_credentials={real_user=slave}} blocked reactor thread for 48097.6us
I20260807 19:50:21.149484 9019 raft_consensus.cc:2468] T b8f9a221ee034d3b86101183048438ff P a38f4d6088cf4c8cb510200ca0555801 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 671d8754170646eaac5748745eba58b4 in term 7.
I20260807 19:50:21.099411 13950 raft_consensus.cc:2749] T 8666b7cb7a05408a8223317d32823bc9 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:21.105414 13967 raft_consensus.cc:2749] T 7abc25df9cc247e8b0354c9758c11208 P a38f4d6088cf4c8cb510200ca0555801 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260807 19:50:21.154778 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 73784.8us
I20260807 19:50:21.157104 8915 raft_consensus.cc:2468] T 17510c7d48eb44ceb3d8c645ceaa28fc P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 7.
I20260807 19:50:21.105084 13823 raft_consensus.cc:493] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:21.131560 13819 raft_consensus.cc:697] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [term 10 LEADER]: Becoming Leader. State: Replica: 671d8754170646eaac5748745eba58b4, State: Running, Role: LEADER
I20260807 19:50:21.135951 13948 consensus_queue.cc:1048] T 90fa71b3b19f407d8ede2571544c265c P 8e9ad2b5e2a94d4faff90af735c6d792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260807 19:50:21.144589 13660 consensus_queue.cc:237] T d8160be756c34002a2ba00cf8deaa10a P a38f4d6088cf4c8cb510200ca0555801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.100539 13975 raft_consensus.cc:493] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 39975243456d4452b4a4870ac7ebafc5)
I20260807 19:50:21.150605 13959 raft_consensus.cc:493] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:50:21.167655 8906 raft_consensus.cc:2468] T fced846726734919ac0881f9a26379ba P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9ad2b5e2a94d4faff90af735c6d792 in term 6.
I20260807 19:50:21.152753 13973 raft_consensus.cc:2749] T ea5ba8c8385a404799a0be85834f98f4 P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260807 19:50:21.171015 13890 leader_election.cc:290] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Requested vote from peers 8e9ad2b5e2a94d4faff90af735c6d792 (127.6.42.193:33185), 39975243456d4452b4a4870ac7ebafc5 (127.6.42.196:35143)
W20260807 19:50:21.161608 13968 log.cc:927] Time spent T fbfe42e0961647279ca7973da14c62a0 P a38f4d6088cf4c8cb510200ca0555801: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260807 19:50:21.178150 9892 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e005b1fcb3134fa9a076533e372b80f9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "39975243456d4452b4a4870ac7ebafc5"
I20260807 19:50:21.183354 9892 raft_consensus.cc:3060] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 7 FOLLOWER]: Advancing to term 8
I20260807 19:50:21.184700 13819 consensus_queue.cc:237] T 301b269c068f46ebae7e1c1d55e501f1 P 671d8754170646eaac5748745eba58b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.185855 8776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8666b7cb7a05408a8223317d32823bc9" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:21.186537 8776 raft_consensus.cc:2468] T 8666b7cb7a05408a8223317d32823bc9 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:21.189225 8893 tablet_service.cc:1558] Processing DeleteTablet for tablet a898110875ba40ab8e512e124aff4fb2 with delete_type TABLET_DATA_TOMBSTONED (TS 671d8754170646eaac5748745eba58b4 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:39536
I20260807 19:50:21.195588 9892 raft_consensus.cc:2468] T e005b1fcb3134fa9a076533e372b80f9 P 39975243456d4452b4a4870ac7ebafc5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 8.
I20260807 19:50:21.177326 8972 leader_election.cc:304] T b89518940e0d4f7c848bc64e87de7e25 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 8e9ad2b5e2a94d4faff90af735c6d792
I20260807 19:50:21.165496 13823 raft_consensus.cc:515] T 3def68aebfd04eeab681e41691bae18c P 671d8754170646eaac5748745eba58b4 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.208393 8970 leader_election.cc:304] T e005b1fcb3134fa9a076533e372b80f9 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39975243456d4452b4a4870ac7ebafc5, a38f4d6088cf4c8cb510200ca0555801; no voters:
I20260807 19:50:21.218688 8789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7572f4807051425c936e926fa8c8d74e" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:21.219632 8789 raft_consensus.cc:2468] T 7572f4807051425c936e926fa8c8d74e P 8e9ad2b5e2a94d4faff90af735c6d792 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 12.
W20260807 19:50:21.267426 8970 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.196:35143, user_credentials={real_user=slave}} blocked reactor thread for 59144.5us
I20260807 19:50:21.178900 13959 raft_consensus.cc:515] T d8160be756c34002a2ba00cf8deaa10a P 671d8754170646eaac5748745eba58b4 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" member_type: VOTER last_known_addr { host: "127.6.42.193" port: 33185 } } peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } }
I20260807 19:50:21.273906 8775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bff76e2c6d6f4e35bf33640e6f4312a0" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:21.275538 8775 raft_consensus.cc:2468] T bff76e2c6d6f4e35bf33640e6f4312a0 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 10.
I20260807 19:50:21.277314 8780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80eb998e349c4e2885504025672242a7" candidate_uuid: "a38f4d6088cf4c8cb510200ca0555801" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8e9ad2b5e2a94d4faff90af735c6d792" is_pre_election: true
I20260807 19:50:21.278759 8780 raft_consensus.cc:2468] T 80eb998e349c4e2885504025672242a7 P 8e9ad2b5e2a94d4faff90af735c6d792 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a38f4d6088cf4c8cb510200ca0555801 in term 6.
I20260807 19:50:21.206135 13973 raft_consensus.cc:493] T 479d60d809ec458397b1360e820ed150 P 671d8754170646eaac5748745eba58b4 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260807 19:50:21.290048 8972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.42.194:37659, user_credentials={real_user=slave}} blocked reactor thread for 121386us
W20260807 19:50:21.284847 9850 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260807 19:50:21.291760 8972 leader_election.cc:304] T 300f918f185249e6895ab57edd060d31 P a38f4d6088cf4c8cb510200ca0555801 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 671d8754170646eaac5748745eba58b4, a38f4d6088cf4c8cb510200ca0555801; no voters: 39975243456d4452b4a4870ac7ebafc5
I20260807 19:50:21.183310 13975 raft_consensus.cc:515] T 413683c7378447239874762d812d75e5 P 671d8754170646eaac5748745eba58b4 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "671d8754170646eaac5748745eba58b4" member_type: VOTER last_known_addr { host: "127.6.42.194" port: 37659 } } peers { permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195" port: 35801 } } peers { permanent_uuid: "39975243456d4452b4a4870ac7ebafc5" member_type: VOTER last_known_addr { host: "127.6.42.196" port: 35143 } attrs { promote: false } }
I20260807 19:50:21.178830 13950 consensus_queue.cc:1059] T 8977afab19724738a872c7880e363557 P 671d8754170646eaac5748745eba58b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a38f4d6088cf4c8cb510200ca0555801" member_type: VOTER last_known_addr { host: "127.6.42.195"